Sylvain Gelly
Marc Schoenauer
Abstract
The ancient oriental game of Go has long been considered a grand challenge for artificial intelligence. For decades, computer Go has defied the classical methods in game tree search that worked so successfully for chess and checkers. However, recent play in computer Go has been transformed by a new paradigm for tree search based on Monte-Carlo methods. Programs based on Monte-Carlo tree search now play at human-master levels and are beginning to challenge top professional players. In this paper, we describe the leading algorithms for Monte-Carlo tree search and explain how they have advanced the state of the art in computer Go.
- Special issue on Monte Carlo techniques and computer Go. In C.-S. Lee, M. Müller, and O. Teytaud, eds, IEEE Trans. Comput. Intell. AI in Games, 2 (2010).Google Scholar
- Abramson, B. Expected-outcome: a general model of static evaluation. IEEE Trans. Patt. Anal. Mach. Intell. 12 (1990), 182--193. Google ScholarDigital Library
- Auer, P., Cesa-Bianchi, N., Fischer, P. Finite time analysis of the multiarmed bandit problem. Mach. Learn. 47(2--3) (2002), 235--256. Google ScholarDigital Library
- Bourki, A., Chaslot, G., Coulm, M., Danjean, V., Doghmen, H., Hoock, J.B., Herault, T., Rimmel, A., Teytaud, F., Teytaud, O., Vayssiere, P., Yu, Z. Scalability and parallelization of Monte-Carlo tree search. In 7th International Conference on Computers and Games (CG-10) (2010), 48--58. Google ScholarDigital Library
- Bouzy, B., Chaslot, G. Bayesian generation and integration of k-nearest-neighbor patterns for 19 × 19 Go. In IEEE Symposium on Computational Intelligence and Games (CIG-05) (2005).Google Scholar
- Bouzy, B., Helmstetter, B. Monte-Carlo Go developments. In 10th International Conference on Advances in Computer Games (ACG-03) (2003), 159--174.Google Scholar
- Brügmann, B. Monte-Carlo Go. Technical report, Max Planck Institute of Physics, 1993.Google Scholar
- Bubeck, S., Munos, R., Stoltz, G., Szepesvári, C. Online optimization in X-armed bandits. In Advances in Neural Information Processing Systems 22 (NIPS-22), D. Koller and D. Schuurmans and Y. Bengio and L. Bottou, eds. MIT Press, 2009, 201--208.Google Scholar
- Cazenave, T., Balbo, F., Pinson, S. Monte-Carlo bus regulation. In 12th International IEEE Conference on Intelligent Transportation Systems (2009), 340--345.Google Scholar
- Chevelu, J., Putois, G., Lepage, Y. The true score of statistical paraphrase generation. In 23rd International Conference on Computational Linguistics: Posters (2010), 144--152. Google ScholarDigital Library
- Coquelin, P.A., Munos, R. Bandit algorithms for tree search. In 23rd Conference on Uncertainty in Artificial Intelligence (UAI-07) (2007), 67--74.Google Scholar
- Coulom, R. Efficient selectivity and backup operators in Monte-Carlo tree search. In 5th International Conference on Computers and Games (CG-06) (2006), 72--83. Google ScholarDigital Library
- Coulom, R. Computing Elo ratings of move patterns in the game of Go. Int. Comput. Game. Assoc. J. 30, 4 (2007), 198--208.Google Scholar
- Finnsson, H., Björnsson, Y. Simulation-based approach to general game playing. In 23rd AAAI Conference on Artificial Intelligence (AAAI-08) (2008), 259--264. Google ScholarDigital Library
- Gelly, S., Silver, D. Monte-Carlo tree search and rapid action value estimation in computer Go. Artif. Intell. 175 (2011), 1856--1875. Google ScholarDigital Library
- Gelly, S., Wang, Y., Munos, R., Teytaud, O. Modification of UCT with Patterns in Monte-Carlo Go. Rapport de recherche INRIA RR-6062, 2006.Google Scholar
- Huang, S., Coulom, R., Lin, S. Monte-Carlo simulation balancing in practice. In 7th International Conference on Computers and, Games (CG-09) (2009), 119--126. Google ScholarDigital Library
- Kocsis, L., Szepesvári, C. Bandit based Monte-Carlo planning. In 15th European Conference on Machine Learning (ECML) (2006), 282--293. Google ScholarDigital Library
- Lai, T.L., Robbins, H. Asymptotically efficient adaptive allocation rules. Adv. Appl. Math. 6 (1985), 4--22.Google ScholarDigital Library
- Nakhost, H., Müller, M. Monte-Carlo exploration for deterministic planning. In 21st International Joint Conference on Artificial Intelligence (IJCAI-09) (2009), 1766--1771. Google ScholarDigital Library
- Robbins, H. Some aspects of the sequential design of experiments. Bull. Am. Math. Soc. 58 (1952), 527--535.Google ScholarCross Ref
- Schaeffer, J. The games computers (and people) play. Adv. Comput., 52 (2000), 190--268.Google Scholar
- Tanabe, Y., Yoshizoe, K., Imai, H. A study on security evaluation methodology for image-based biometrics authentication systems. In 3rd IEEE International Conference on Biometrics: Theory, Applications and Systems (2009), 258--263. Google ScholarDigital Library
- Widrow, B., Gupta, N.K., Maitra, S. Punish/reward: Learning with a critic in adaptive threshold systems. IEEE Trans. Syst., Man, Cybern. 3 (1973), 455--465.Google ScholarCross Ref
- Zinkevich, M., Bowling, M., Bard, N., Kan, M., Billings, D. Optimal unbiased estimators for evaluating agent performance. In 21st National Conference on Artificial Intelligence (AAAI-06) (2006), 573--578. Google ScholarDigital Library
Index Terms
- The grand challenge of computer Go: Monte Carlo tree search and extensions
Recommendations
Go and the computer
It seems that a disproportionately large number of mathematicians and computer engineers are members of the Go playing population in Western countries, and there is a sound reason for this. The development of Game Theory received its first great boost ...
Computer Go
Chips challenging champions: games, computers and Artificial IntelligenceComputer Go is one of the biggest challenges faced by game programmers. This survey describes the typical components of a Go program, and discusses knowledge representation, search methods and techniques for solving specific subproblems in this domain. ...
Multimedia Grand Challenge 2012
The Multimedia Grand Challenge is a recurring event at the ACM Multimedia Conference series. During this event, delegates from various industries define a number of challenges that they consider of interest from both a business and scientific ...
Comments