Skip to main content
SpringerLink
Log in

An Analysis of Competitive Coevolutionary Particle Swarm Optimizers to Train Neural Network Game Tree Evaluation Functions

  • Conference paper
  • First Online:
Advances in Swarm Intelligence (ICSI 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9712))

Included in the following conference series:

Abstract

Particle swarm optimization (PSO) has been applied in the past to train neural networks (NN) as evaluation functions for zero-sum board games. The NN weights were adjusted using PSO in a competitive coevolutionary approach. Recent analyses of PSO as a NN training algorithm have revealed a serious issue when bounded activation functions are used in the hidden layer of the NNs: Very early during the training process, activation function saturation occurs, at which point weight adjustments stagnate. This paper studies the effect of activation function stagnation on previously used competitive coevolutionary training of NNs using PSO, and shows that the results reported indicates performance similar to making random game moves, and worse than random moves for ply depths larger than one. New results are presented showing more efficient training of NN game tree evaluation functions when unbounded activation functions are used in the hidden layer, and bounded activation functions in the output layer.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chellapilla, K., Fogel, D.: Evolving neural networks to play checkers without relying on expert knowledge. IEEE Trans. Neural Netw. 10(6), 1382–1391 (1999)

    Article  Google Scholar 

  2. Conradie, J., Engelbrecht, A.: Training Bao game-playing agents using coevolutionary particle swarm optimization. In: IEEE Symposium on Computational Intelligence and Games. pp. 67–74 (2006)

    Google Scholar 

  3. Engelbrecht, A.P.: Computational Intelligence: An Introduction, 2nd edn. Wiley Publishing, Hoboken (2007)

    Book  Google Scholar 

  4. Engelbrecht, A.: Particle swarm optimization: velocity initialization. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 1–8 (2012)

    Google Scholar 

  5. Franken, C.J.: PSO-based Coevolutionary Game Learning. Master’s thesis (2004)

    Google Scholar 

  6. Franken, N., Engelbrecht, A.: Comparing PSO structures to learn the game of checkers from zero knowledge. In: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 1, pp. 234–241 (2003)

    Google Scholar 

  7. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the International Joint Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)

    Google Scholar 

  8. Messerschmidt, L., Engelbrecht, A.: Learning to play games using a PSO-based competitive learning approach. IEEE Trans. Evol. Comput. 8(3), 280–288 (2004)

    Article  Google Scholar 

  9. Schaeffer, J., Lake, R.: Solving the game of checkers. Games No Chance 29, 119–133 (1996)

    MathSciNet  MATH  Google Scholar 

  10. Scheepers, C., Engelbrecht, A.: Analysis of stagnation behaviour of competitive coevolutionary trained neuro-controllers. In: Proceedings of the IEEE Swarm Intelligence Symposium (2014)

    Google Scholar 

  11. Scheepers, C., Engelbrecht, A.: Training multi-agent teams from zero knowledge with the competitive coevolutionary team-based particle swarm optimiser. Soft Comput. 20(2), 607–620 (2016)

    Article  Google Scholar 

  12. WCDF: Rules of Draughts (Checkers) (2015). http://www.wcdf.net/rules.htm. (Accessed on 19 October 2015)

  13. van Wyk, A., Engelbrecht, A.: Overfitting by PSO trained feedforward neural networks. In: Proceedings of the IEEE Congress on Evolutionary Computation (2010)

    Google Scholar 

  14. van Wyk, A., Engelbrecht, A.: Lamda-Gamma learning with feedforward neural networks using particle swarm optimization. In: Proceedings of the IEEE Swarm Intelligence Symposium (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Pretoria, Pretoria, South Africa

    Albert Volschenk & Andries Engelbrecht

Authors
  1. Albert Volschenk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andries Engelbrecht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andries Engelbrecht .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Ying Tan

  2. Xi'an Jiaotong-Liverpool University, Suzhou, China

    Yuhui Shi

  3. Shenzhen University, Shenzhen, China

    Ben Niu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Volschenk, A., Engelbrecht, A. (2016). An Analysis of Competitive Coevolutionary Particle Swarm Optimizers to Train Neural Network Game Tree Evaluation Functions. In: Tan, Y., Shi, Y., Niu, B. (eds) Advances in Swarm Intelligence. ICSI 2016. Lecture Notes in Computer Science(), vol 9712. Springer, Cham. https://doi.org/10.1007/978-3-319-41000-5_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41000-5_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-40999-3

  • Online ISBN: 978-3-319-41000-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation