Abstract
In Chinese-chess computer game (CCCG), the position evaluation function plays a crucial role in building a game playing program. Traditionally, there are two typical types of evaluation functions: standard heuristic evaluation function (SHEF) and self learning evaluation function (SLEF). The SHEF depends on the board position features to large extent, but it hardly includes all the features due to the limit of knowledge of the designer. The SLEF can explore the knowledge hidden in the current position which is difficult to find in the SHEF. In this paper, a combined position evaluation function (CPEF) is designed by weighted sum of SHEF and SLEF. SHEF considers the material balance and adjunctive value of position while SLEF takes the form of a three-layer fully-connected feed forward neural network. We use temporal difference learning (TDL) to train the neural network on professional game records. Based on the combined position evaluation function, a Chinese chess program HBUCHESS is developed. We experimentally validate that our CPEF is quite effective through competing with different kinds of testing players. With the help of CPEF, the intelligent level of HBUCHESS can be improved incrementally with the increase of number of professional game records SLEF learned. Furthermore, in the process of learning professional game records, we find that the performance of HBUCHESS is mainly relevant to the following four aspects: (1) the initial heuristic knowledge, (2) the number of nodes in hidden layer of neural network, (3) the trace decay parameter λ, and (4) the learning rate α.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Shannon, C.E.: Programming a Computer for Playing Chess. Philos. Mag. 41, 256–275 (1950)
Gillogly, J.J.: The Technology Chess Program. Artif. Intell. 3, 145–163 (1972)
Tesauro, G.: TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play. Neural Comput. 6, 215–219 (1994)
Samuel, A.L.: Some Studies in Machine Learning Using the Game of Checkers. IBM J. Res. Dev. 3, 210–229 (1959)
Sutton, R.S.: Learning to Predict by the Methods of Temporal Differences. Mach. Learn. 3, 9–44 (1988)
Tesauro, G.: Practical Issues in Temporal Difference Learning. Mach. Learn. 8, 257–277 (1992)
Tesauro, G.: Temporal Difference Learning and TD-Gammon. Commun. ACM 38, 58–68 (1995)
Schraudolph, N.N., Dayan, P., Sejnowski, T.J.: Temporal Difference Learning of Position Evaluation in the Game of Go. Adv. Neur. Infor. Proc. Syst. 6, 817–824 (1994)
Enzenberger, M.: The Integration of a Priori Knowledge into a Go Playing Neural Network (1996), http://www.cgl.ucsf.edu/go/Programs/neurogohtml/neurogo.html
Baxter, J., Trigdell, A., Weaver, L.: Knightcap: a Chess Program that Learns by Combining TD(λ) with Game-Tree Search. In: Proc. Inter. Conf. Mach. Learn., pp. 28–36 (1998)
Baxter, J., Tridgell, A., Weaver, L.: Learning To Play Chess Using Temporal Differences. Mach. Learn. 40, 243–263 (2001)
Trinh, T.B., Bashi, A.S., Deshpande, N.: Temporal Difference Learning in Chinese Chess. In: Mira, J., Moonis, A., de Pobil, A.P. (eds.) IEA/AIE 1998. LNCS, vol. 1416, pp. 612–618. Springer, Heidelberg (1998)
Olson, D.K.: Learning to Play Games from Experience: An Application of Artificial Neural Networks and Temporal Difference Learning. Master’s thesis. Pacific Lutheran University, Washington (1993)
Thrun, S.: Learning to Play the Game of Chess. Adv. Neur. Infor. Proc. Syst. 7, 1069–1076 (1995)
Chellapilla, K., Fogel, D.: Evolving Neural Network to Play Checkers without Expert Knowledge. IEEE Trans. Neur. Netw. 10, 1382–1391 (1999)
Schaeffer, J., Hlynak, M., Jussila, V.: Temporal Difference Learning Applied to a High-Performance Game-Playing Program. In: Inter. Joint Conf. Artif. Intell., pp. 529–534 (2001)
Ghory, I.: Reinforcement Learning in Board Games. Technical Report: CSTR-04-004, Department of Computer Science, University of Bristol (2004)
Wiering, M.A., Patist, J.P.: Learning to Play Board Games using Temporal Difference Methods. Technical Report: UU-CS-2005-048. Institute of Information and Computing Sciences, Utrecht University (2005)
Tu, Z.J.: Design and Implementation of Computer Chess. Master thesis. Zhongshan University (2004)
Xu, X.H., Jiao Wang, J.: Key Technologies Analysis of Chinese Chess Computer Game. Mini-Micro Syst. 27, 961–969 (2006)
Wei, C.B.: Study and Implementation of Chinese Chess Computer Game System. Master thesis. Kunming University of Science and Technology (2008)
Qiang, F.: The Research of Chinese Chess Based On Reinforcement Learning. Master thesis. Changsha University of Science and Technology (2006)
Wang, Y.F.: Research and Implementation of a Chinese Chess Computer Game System with Self Learning. Master thesis. Harbin Engineering University (2007)
Mitchell, T.M.: Machine Learning. The McGraw-Hill Companies, Inc. (1997)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. J. of Artif. Intell. Res. 4, 237–285 (1996)
Lau, H.T.: Chinese Chess: An Introduction to China’s Ancient Game of Strategy. Tuttle Publishing (2003)
Greenblatt, R.D., Eastlake, D.E., Crocker, S.D.: The Greenblatt Chess Program. In: Fall Joint Compu. Conf. Proc., pp. 801–810 (1967)
Hsu, S.C.: Past and Future of Computer Chess and Computer Chinese Chess. J. Compu. 2, 1–8 (1990)
Knuth, D.E., Moore, R.W.: An Analysis of Alpha-Beta Pruning. Artif. Intell. 6, 293–326 (1975)
Schaeffer, J.: The History Heuristic and Alpha-Beta Search Enhancement in Practice. IEEE Trans. Pattern Anal. Mach. Intell. 11, 1203–1212 (1989)
Marsland, T.A.: A Review of Game-Tree Pruning. ICCA Journal 9, 3–19 (2001)
Su, P., Li, Y., Li, Y.J., Shiu, S.C.K.: An auto-adaptive convex map generating path-finding algorithm: Genetic Convex A*. Inter. J. Mach. Learn. & Cyber. (2012), doi: 10.1007/s13042-012-0120-x
Chen, W.Y., Heish, S.Y., Yen, C.Y., Kuo, D.Y.: The Chinese-Chess Image Identification Techniques on Spatial Domain. In: Proc. 2011 World Congr. Intell. Control and Autom., pp. 970–974 (2011)
Su, K.L., Chung, C.Y., Zou, J.T., Hsu, T.Y.: A* Search Algorithm Applied to a Chinese Chess Game. Artif. Life & Robot. 16, 132–136 (2011)
Chen, B.N., Liu, P.F., Hsu, S.C., Hsu, T.S.: Aggregating Consistent endgame knowledge in Chinese Chess. Knowledge-Based Syst. (2012), doi: 10.1016/j.knosys.2011.11.020
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
He, Y., Wang, X., Fu, T. (2013). A Combined Position Evaluation Function in Chinese Chess Computer Game. In: Gavrilova, M.L., Tan, C.J.K. (eds) Transactions on Computational Science XVII. Lecture Notes in Computer Science, vol 7420. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35840-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-35840-1_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35839-5
Online ISBN: 978-3-642-35840-1
eBook Packages: Computer ScienceComputer Science (R0)