Abstract
The learning dynamics close to the initial conditions of an on-line Hebbian ICA algorithm has been studied. For large input dimension the dynamics can be described by a diffusion equation.A surprisingly large number of examples and unusually low initial learning rate are required to avoid a stochastic trapping state near the initial conditions. Escape from this state results in symmetry breaking and the algorithm therefore avoids trapping in plateau-like fixed points which have been observed in other learning algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Hyvärinen, A.: Survey on independent component analysis. Neural Computing Surveys 2 (1999) 94–128
Saad, D. and Solla, S. A.: Exact solution for on-line learning in multilayer neural networks. Physical Review Letters E 74 (1995) 4337–4340.
Biehl, M., Schlosser, E.: The dynamics of on-line principal component analysis. Journal of Physics A31 (1998) L97
Rattray, M.: Stochastic trapping in a solvable model of on-line independent component analysis. Neural Computation 14,2 (2002) 421–435
Rattray, M., Basalyga, G.: Scaling laws and local minima in Hebbian ICA. To appear in Proceedings of Neural Information Processing Systems (NIPS*2001), Vancouver (2002)
Hyvärinen A., Oja, E.: Signal Processing, 64 (1998) 301–313
Gardiner, C.W.: Handbook of Stochastic Methods. Springer-Verlag, NewYork(1985)
Van Kampen, N.G.: Stochastic processes in physics and chemistry. Elsevier, Amsterdam(1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Basalyga, G., Rattray, M. (2002). Dynamics of ICA for High- Dimensional Data. In: Dorronsoro, J.R. (eds) Artificial Neural Networks — ICANN 2002. ICANN 2002. Lecture Notes in Computer Science, vol 2415. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46084-5_180
Download citation
DOI: https://doi.org/10.1007/3-540-46084-5_180
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44074-1
Online ISBN: 978-3-540-46084-8
eBook Packages: Springer Book Archive