Abstract
We review a new form of immediate reward reinforcement learning in which the individual unit is deterministic but has stochastic synapses. 4 learning rules have been developed from this perspective and we investigate the use of these learning rules to perform linear projection techniques such as principal component analysis, exploratory projection pursuit and canonical correlation analysis. The method is very general and simply requires a reward function which is specific to the function we require the unit to perform. We also discuss how the method can be used to learn kernel mappings and conclude by illustrating its use on a topology preserving mapping.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bishop, C.M., Svensen, M., Williams, C.K.I.: Gtm: The generative topographic mapping. Neural Computation (1997)
Fyfe, C.: Two topographic maps for data visualization. Data Mining and Knowledge Discovery (2007) ISSN 1384-5810
Fyfe, C., Lai, P.L.: Immediate reward reinforcement learning for projective kernel methods. In: ESANN 2007. 14th European Symposium on Artificial Neural Networks (2007)
Fyfe, C., Lai, P.L.: Reinforcement learning reward functions for unsupervised learning. In: ISNN 2007. 4th International Symposium on Neural Networks (2007)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Kohonen, T.: Self-Organising Maps. Springer, Heidelberg (1995)
Lai, P.L., Fyfe, C.: Reinforcement learning for topographic mappings. In: 20th International Joint Conference on Artificial Neural Networks (2007)
Lai, P.L., Fyfe, C.: Kernel and nonlinear canonical correlation analysis. International Journal of Neural Systems 10(5), 365–377 (2001)
Likas, A.: A reinforcement learning approach to on-line clustering. Neural Computation (2000)
Ma, X., Likharev, K.K.: Global reinforcement learning in neural networks with stochastic synapses. IEEE Transactions on Neural Networks 18(2), 573–577 (2007)
Scholkopf, B., Smola, A., Muller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: an Introduction. MIT Press, Cambridge (1998)
Williams, R.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256 (1992)
Williams, R.J., Pong, J.: Function optimization using connectionist reinforcement learning networks. Connection Science 3, 241–268 (1991)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, Y., Fyfe, C., Lai, P.L. (2007). Stochastic Weights Reinforcement Learning for Exploratory Data Analysis. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_68
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_68
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)