Skip to main content
SpringerLink
Log in

Evaluation-function modeling with multi-layered perceptron for RoboCup soccer 2D simulation

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

In the RoboCup soccer simulation 2D league, players make a decision at each cycle in real time. The performance of a team highly depends on the agents’ decision-making process, which is composed of a action planning method and an evaluation function of the soccer field. In this work, a cooperative action planning based on the tree search is employed. Each action is evaluated by an evaluation function. We employ a multi-layered perceptron to construct an evaluation function. We examine the performance of the soccer agents when various sets of features are used as the input of the neural network. A feature vector is made of kick sequences executed by an expert team extracted from log files. To investigate the efficiency of our approach, we compare the performance of a team using an evaluation function modeled by neural networks against a team using a hand-tuned evaluation function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Silver D, Hubert T, Schrittwieser J et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Nature 362(6419):1140–1144

    MathSciNet  MATH  Google Scholar 

  2. Kitano H, Asada M, Kuniyoshi Y et al (1997) RoboCup: a challenge problem for AI and robotics. In: RoboCup-97: Robot Soccer World Cup I, Springer, Berlin, pp 1–19

    Google Scholar 

  3. Akiyama H, Aramaki S, Nakashima T (2012) Online cooperative behavior planning using a tree search method in the RoboCup soccer simulation. In: Proceedings of 4th IEEE international conference on intelligent networking and collaborative systems, pp 170–177

  4. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Networks 61:85–117

    Article  Google Scholar 

  5. Liu W, Wang Z, Liu X et al (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26

    Article  Google Scholar 

  6. Warnell G, Waytowich N, Lawhern V et al (2018) Deep TAMER: interactive agent shaping in high-dimensional state spaces. In: Proceedings of the 32nd AAAI conference on artificial intelligence (AAAI-18), pp 1545–1554

  7. Stanescu M, Barriga NA, Hess A et al (2016) Evaluating real-time strategy game states using convolutional neural networks. In: Proceedings of the IEEE conference on computational intelligence and games, pp 1–7

  8. Hong Z, Su S, Shann T et al (2018) A deep policy inference Q-network for multi-agent systems. In: Proceedings of the 17th international conference on autonomous agents and multiagent systems (AAMAS’18), pp 1388–1396

  9. Floyd MW, Esfandiari B, Lam K (2008) A case-based reasoning approach to imitating RoboCup players. In: Proceedings of the 21st international FLAIRS conference, pp 251–256

  10. Hausknecht M, Stone P (2016) Deep reinforcement learning in parameterized action space. In: Proceedings of the international conference on learning representations (ICLR), pp 1–12

  11. Liu Y, Stone P (2006) Value-function-based transfer for reinforcement learning using structure mapping. In: Proceedings of the 21st national conference on artificial intelligence, pp 415–420

  12. Akiyama H, Nakashima T (2013) Helios base: an open source package for the robocup soccer 2D simulation. In: RoboCup 2012: Robot Soccer World Cup XVI, Springer, Berlin, pp 528–535

    Google Scholar 

  13. Sugino T, Ito T, Arimura Y et al (2018) Team HillStone2018 in the 2DSimulation league team description paper. In: RoboCup2018, Montreal, Canada, pp 1–6

  14. Akiyama H, Nakashima T, Suzuki Y et al (2018) HELIOS2018: Team description paper. In: RoboCup2018, Montreal, Canada, pp 1–6

  15. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the 14th international conference on artificial intelligence and statistics, pp 315–323

  16. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the 3rd international conference on learning representations (ICLR), pp 1–15

Download references

Author information

Authors and Affiliations

  1. Osaka Prefecture University, Sakai, Osaka, Japan

    Takuya Fukushima & Tomoharu Nakashima

  2. Fukuoka University, Fukuoka, Fukuoka, Japan

    Hidehisa Akiyama

Authors
  1. Takuya Fukushima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomoharu Nakashima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hidehisa Akiyama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Takuya Fukushima.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fukushima, T., Nakashima, T. & Akiyama, H. Evaluation-function modeling with multi-layered perceptron for RoboCup soccer 2D simulation. Artif Life Robotics 25, 440–445 (2020). https://doi.org/10.1007/s10015-020-00602-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-020-00602-w

Keywords

Search

Navigation