skip to main content
10.1145/2463372.2463512acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Improving coevolution by random sampling

Published: 06 July 2013 Publication History

Abstract

Recent developments cast doubts on the effectiveness of coevolutionary learning in interactive domains. A simple evolution with fitness evaluation based on games with random strategies has been found to generalize better than competitive coevolution. In an attempt to investigate this phenomenon, we analyze the utility of random opponents for one and two-population competitive coevolution applied to learning strategies for the game of Othello. We show that if coevolution uses two-population setup and engages also random opponents, it is capable of producing equally good strategies as evolution with random sampling for the expected utility performance measure. To investigate the differences between analyzed methods, we introduce performance profile, a tool that measures the player's performance against opponents of various strength. The profiles reveal that evolution with random sampling produces players coping well with mediocre opponents, but playing relatively poorly against stronger ones. This finding explains why in the round-robin tournament, evolution with random sampling is one of the worst methods from all those considered in this study.

References

[1]
R. Axelrod. The evolution of strategies in the iterated prisoner's dilemma. In L. Davis, editor, Genetic Algorithms in Simulated Annealing, pages 32--41. Pitman, London, 1987.
[2]
K. J. Binkley, K. Seehart, and M. Hagiwara. A Study of Artificial Neural Network Architectures for Othello Evaluation Functions. Transactions of the Japanese Society for Artificial Intelligence, 22(5):461--471, 2007.
[3]
M. Buro. The evolution of strong othello programs. Entertainment Computing-Technology and Applications, pages 81--88, 2003.
[4]
S. Y. Chong, M. K. Tan, and J. D. White. Observing the evolution of neural networks learning to play the game of othello. Evolutionary Computation, IEEE Transactions on, 9(3):240--251, 2005.
[5]
S. Y. Chong, P. Tino, D. C. Ku, and Y. Xin. Improving Generalization Performance in Co-Evolutionary Learning. IEEE Transactions on Evolutionary Computation, 16(1):70--85, 2012.
[6]
E. D. de Jong. The maxsolve algorithm for coevolution. In H.-G. B. et al., editor, GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, volume 1, pages 483--489, Washington DC, USA, 25--29 June 2005. ACM Press.
[7]
E. D. de Jong. A Monotonic Archive for Pareto-Coevolution. Evolutionary Computation, 15(1):61--93, Spring 2007.
[8]
E. D. de Jong and A. Bucci. DECA: dimension extracting coevolutionary algorithm. In M. C. et al., editor, GECCO 2006: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pages 313--320, Seattle, Washington, USA, 2006. ACM Press.
[9]
S. G. Ficici. Solution concepts in coevolutionary algorithms. PhD thesis, Waltham, MA, USA, 2004. Adviser-Pollack, Jordan B.
[10]
S. G. Ficici. Multiobjective Optimization and Coevolution. In J. Knowles, D. Corne, and K. Deb, editors, Multi-Objective Problem Solving from Nature: From Concepts to Applications, pages 31--52. Springer, Berlin, 2008.
[11]
S. G. Ficici and J. B. Pollack. Pareto optimality in coevolutionary learning. In J. Kelemen and P. Sosík, editors, Advances in Artificial Life, 6th European Conference, ECAL 2001, volume 2159 of Lecture Notes in Computer Science, pages 316--325, Prague, Czech Republic, 2001. Springer.
[12]
D. B. Fogel. Blondie24: Playing at the Edge of AI. Morgan Kaufmann Publishers, Sept. 2001.
[13]
W. Jaskowski, K. Krawiec, and B. Wieloch. Evolving strategy for a probabilistic game of imperfect information using genetic programming. Genetic Programming and Evolvable Machines, 9(4):281--294, 2008.
[14]
W. Jaskowski, K. Krawiec, and B. Wieloch. Winning ant wars: Evolving a human-competitive game strategy using fitnessless selection. In M. O. et al., editor, Genetic Programming 11th European Conference, EuroGP 2008, Proceedings, volume 4971 of Lecture Notes in Computer Science, pages 13--24. Springer-Verlag, mar 2008.
[15]
H. Juillé and J. B. Pollack. Coevolving the "ideal" trainer: Application to the discovery of cellular automata rules. In University of Wisconsin, pages 519--527. Morgan Kaufmann, 1998.
[16]
K. Krawiec, W. Ja\'skowski, and M. Szubert. Evolving small-board go players using coevolutionary temporal difference learning with archive. International Journal of Applied Mathematics and Computer Science, 21(4):717--731, 2011.
[17]
S. Lucas and T. P. Runarsson. Othello Competition; http://algoval.essex.ac.uk:8080/othello/League.jsp.
[18]
S. M. Lucas. Learning to play Othello with N-tuple systems. Australian Journal of Intelligent Information Processing Systems, Special Issue on Game Technology, 9(4):1--20, 2007.
[19]
S. M. Lucas and T. P. Runarsson. Temporal difference learning versus co-evolution for acquiring othello position evaluation. In IEEE Symposium on Computational Intelligence and Games, pages 52--59. IEEE, 2006.
[20]
S. Luke and R. P. Wiegand. When coevolutionary algorithms exhibit evolutionary dynamics. In A. M. Barry, editor, GECCO 2002: Proceedings of the Bird of a Feather Workshops, Genetic and Evolutionary Computation Conference, pages 236--241, New York, 2002.
[21]
J. Mandziuk. Knowledge-Free and Learning-Based Methods in Intelligent Game Playing, volume 276. Springer, 2010.
[22]
E. P. Manning. Using resource-limited nash memory to improve an othello evaluation function. Computational Intelligence and AI in Games, IEEE Transactions on, 2(1):40--53, 2010.
[23]
J. B. Pollack and A. D. Blair. Co-evolution in the successful learning of backgammon strategy. Machine Learning, 32(3):225--240, 1998.
[24]
E. Popovici, A. Bucci, R. P. Wiegand, and E. D. de Jong. Handbook of Natural Computing, chapter Coevolutionary Principles. Springer-Verlag, 2011.
[25]
P. Rosenbloom. A world-championship-level othello program. Artificial Intelligence, 19(3):279--320, 1982.
[26]
C. D. Rosin and R. K. Belew. New methods for competitive coevolution. Evolutionary Computation, 5(1):1--29, 1997.
[27]
M. Szubert, W. Jaskowski, and K. Krawiec. Coevolutionary temporal difference learning for othello. In IEEE Symposium on Computational Intelligence and Games, pages 104--111, Milano, Italy, 2009.
[28]
S. van den Dries and M. A. Wiering. Neural-Fitted TD-Leaf Learning for Playing Othello With Structured Neural Networks. IEEE Transactions on Neural Networks and Learning Systems, 23(11):1701--1713, Nov. 2012.
[29]
T. Yoshioka, S. Ishii, and M. Ito. Strategy acquisition for the game "othello" based on reinforcement learning. In ICONIP, pages 841--844, 1998.

Cited By

View all
  • (2025)Global progress in competitive co-evolution: a systematic comparison of alternative methodsFrontiers in Robotics and AI10.3389/frobt.2024.147088611Online publication date: 21-Jan-2025
  • (2017)Online discovery of search objectives for test-based problemsEvolutionary Computation10.1162/evco_a_0017925:3(375-406)Online publication date: 1-Sep-2017
  • (2017)Accelerating coevolution with adaptive matrix factorizationProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071320(457-464)Online publication date: 1-Jul-2017
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '13: Proceedings of the 15th annual conference on Genetic and evolutionary computation
July 2013
1672 pages
ISBN:9781450319638
DOI:10.1145/2463372
  • Editor:
  • Christian Blum,
  • General Chair:
  • Enrique Alba
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 July 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. competitive coevolution
  2. maximization of expected utility
  3. othello
  4. performance profile
  5. solution concepts
  6. strategy learning

Qualifiers

  • Research-article

Conference

GECCO '13
Sponsor:
GECCO '13: Genetic and Evolutionary Computation Conference
July 6 - 10, 2013
Amsterdam, The Netherlands

Acceptance Rates

GECCO '13 Paper Acceptance Rate 204 of 570 submissions, 36%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)1
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Global progress in competitive co-evolution: a systematic comparison of alternative methodsFrontiers in Robotics and AI10.3389/frobt.2024.147088611Online publication date: 21-Jan-2025
  • (2017)Online discovery of search objectives for test-based problemsEvolutionary Computation10.1162/evco_a_0017925:3(375-406)Online publication date: 1-Sep-2017
  • (2017)Accelerating coevolution with adaptive matrix factorizationProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071320(457-464)Online publication date: 1-Jul-2017
  • (2017)Social trends in the iterated prisoner's dilemmaProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3067695.3082037(23-24)Online publication date: 15-Jul-2017
  • (2016)Non-negative Matrix Factorization for Unsupervised Derivation of Search Objectives in Genetic ProgrammingProceedings of the Genetic and Evolutionary Computation Conference 201610.1145/2908812.2908888(749-756)Online publication date: 20-Jul-2016
  • (2016)Evolving Chess-like Games Using Relative Algorithm Performance ProfilesApplications of Evolutionary Computation10.1007/978-3-319-31204-0_37(574-589)Online publication date: 15-Mar-2016
  • (2015)Comparison of Semantic-aware Selection Methods in Genetic ProgrammingProceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation10.1145/2739482.2768505(1301-1307)Online publication date: 11-Jul-2015
  • (2015)Towards generating arcade game rules with VGDL2015 IEEE Conference on Computational Intelligence and Games (CIG)10.1109/CIG.2015.7317941(185-192)Online publication date: Aug-2015
  • (2015)The Role of Behavioral Diversity and Difficulty of Opponents in Coevolving Game-Playing AgentsApplications of Evolutionary Computation10.1007/978-3-319-16549-3_32(394-405)Online publication date: 17-Mar-2015
  • (2014)Multi-Criteria Comparison of Coevolution and Temporal Difference Learning on OthelloApplications of Evolutionary Computation10.1007/978-3-662-45523-4_25(301-312)Online publication date: 29-Nov-2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media