Abstract
We extend a reinforcement learning algorithm which has previously been shown to cluster data. Our extension involves creating an underlying latent space with some pre-defined structure which enables us to create a topology preserving mapping. We investigate different forms of the reward function, all of which are created with the intent of merging local and global information, thus avoiding one of the major difficulties with e.g. K-means which is its convergence to local optima depending on the initial values of its parameters. We also show that the method is quite general and can be used with the recently developed method of stochastic weight reinforcement learning [14].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barbakh, W.: Local versus Global Interactions in Clustering Algorithms. Ph.D thesis, School of Computing, University of the West of Scotland (2008)
Barbakh, W., Fyfe, C.: Clustering with reinforcement learning. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 507–516. Springer, Heidelberg (2007)
Bishop, C.M., Svensen, M., Williams, C.K.I.: Gtm: The generative topographic mapping. Neural Computation (1997)
Friedman, J.H.: Exploratory projection pursuit. Journal of the American Statistical Association 82(397), 249–266 (1987)
Friedman, J.H., Tukey, J.W.: A projection pursuit algorithm for exploratory data analysis. IEEE Transactions on Computers c-23(9), 881–889 (1974)
Fyfe, C.: A scale invariant feature map. Network: Computation in Neural Systems 7, 269–275 (1996)
Fyfe, C.: A comparative study of two neural methods of exploratory projection pursuit. Neural Networks 10(2), 257–262 (1997)
Fyfe, C.: Two topographic maps for data visualization. Data Mining and Knowledge Discovery 14, 207–224 (2007)
Intrator, N.: Feature extraction using an unsupervised neural network. Neural Computation 4(1), 98–107 (1992)
Jones, M.C., Sibson, R.: What is projection pursuit. Journal of The Royal Statistical Society, 1–37 (1987)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Kohonen, T.: Self-Organising Maps. Springer, Heidelberg (1995)
Likas, A.: A reinforcement learning approach to on-line clustering. Neural Computation (2000)
Ma, X., Likharev, K.K.: Global reinforcement learning in neural networks with stochastic synapses. IEEE Transactions on Neural Networks 18(2), 573–577 (2007)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: an Introduction. MIT Press, Cambridge (1998)
Williams, R.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256 (1992)
Williams, R.J., Pong, J.: Function optimization using connectionist reinforcement learning networks. Connection Science 3, 241–268 (1991)
Zhang, B.: Generalized k-harmonic means – boosting in unsupervised learning. Technical report, HP Laboratories, Palo Alto (October 2000)
Zhang, B., Hsu, M., Dayal, U.: K-harmonic means - a data clustering algorithm. Technical report, HP Laboratories, Palo Alto (October 1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Fyfe, C., Barbakh, W. (2009). Immediate Reward Reinforcement Learning for Clustering and Topology Preserving Mappings. In: Biehl, M., Hammer, B., Verleysen, M., Villmann, T. (eds) Similarity-Based Clustering. Lecture Notes in Computer Science(), vol 5400. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01805-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-01805-3_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-01804-6
Online ISBN: 978-3-642-01805-3
eBook Packages: Computer ScienceComputer Science (R0)