Abstract
In this paper we derive a simple reinforcement learning rule based on a more general form of REINFORCE formulation. We test our new rule on both classification and reinforcement problems. The results have shown that although this simple learning rule has a high probability of being stuck in local optimum for the case of classification tasks, it is able to solve some global reinforcement problems (e.g. the cart-pole balancing problem) directly in the continuous space.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hertz, J., Palmer, R.G., Krogh, A.S.: Introduction to the theory of neural computation. Addison-Wesley Pub. Co., Redwood City (1991)
Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256 (1992)
Ma, X., Likharev, K.K.: Global reinforcement learning in neural networks with stochastic synapses. In: Proc. of WCCI/IJCNN’06, pp. 47–53 (2006)
Ma, X., Likharev, K.K.: Global reinforcement learning in neural networks. To be published in IEEE Tran. on Neural Networks (2007)
Türel, Ö., Lee, J.H., Ma, X., Likharev, K.K.: Neuromorphic architectures for nanoelectronic circuits. Int. J. Circ. Theory App. 32(5), 277–302 (2004)
Baxter, J., Bartlett, P.L.: Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research 15, 319–350 (2001)
Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans. Syst., Man, Cybern. 13, 834–846 (1983)
Albus, J.S.: A new approach to manipulator conrol: the cerebellar model articulation controller (CMAC). Trans. of ASME Journal of Dynamic Systems, Measurements, and Control 97(3), 220–227 (1975)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ma, X. (2007). An Extremely Simple Reinforcement Learning Rule for Neural Networks. In: Liu, D., Fei, S., Hou, ZG., Zhang, H., Sun, C. (eds) Advances in Neural Networks – ISNN 2007. ISNN 2007. Lecture Notes in Computer Science, vol 4491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72383-7_51
Download citation
DOI: https://doi.org/10.1007/978-3-540-72383-7_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72382-0
Online ISBN: 978-3-540-72383-7
eBook Packages: Computer ScienceComputer Science (R0)