Abstract
We present a novel extension to Independent Component Analysis (ICA), where the data is generated as the product of two submodels, each of which follow an ICA model, and which combine in a horizontal fashion. This is in contrast to previous nonlinear extensions to ICA which were based on a hierarchy of layers. We apply the product model to natural image patches and report the emergence of localized masks in the additional network layer, while the Gabor features that are obtained in the primary layer change their tuning properties and become less localized. As an interpretation we suggest that the model learns to separate the localization of image features from other properties, since identity and position of a feature are plausibly independent. We also show that the horizontal model can be interpreted as an overcomplete model where the features are no longer independent.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research 37(23), 3311–3325 (1997)
van Hateren, J.H., van der Schaaf, A.: Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B 265, 359–366 (1998)
Jutten, C., Herault, J.: Blind separation of sources part i: An adaptive algorithm based on neuromimetic architecture. Signal Processing 24, 1–10 (1991)
Comon, P.: Independent component analysis – a new concept? Signal Processing 36, 287–314 (1994)
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, Chichester (2001)
Hyvärinen, A., Hoyer, P.: Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces. Neural Computation 12(7), 1705–1720 (2000)
Hyvärinen, A., Hoyer, P., Inki, M.: Topographic independent component analysis. Neural Computation (2001)
Atick, J.J.: Could information theory provide an ecological theory of sensory processing? Network: Computation in neural systems 3, 213–251 (1992)
Schwartz, O., Simoncelli, E.P.: Natural signal statistics and sensory gain control. Nature Neuroscience 4(8), 819–825 (2001)
Karklin, Y., Lewicki, M.S.: A hierarchical bayesian model for learning non-linear statistical regularities in non-stationary natural signals. Neural Computation 17(2), 397–423 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Köster, U., Lindgren, J.T., Gutmann, M., Hyvärinen, A. (2009). Learning Natural Image Structure with a Horizontal Product Model. In: Adali, T., Jutten, C., Romano, J.M.T., Barros, A.K. (eds) Independent Component Analysis and Signal Separation. ICA 2009. Lecture Notes in Computer Science, vol 5441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00599-2_64
Download citation
DOI: https://doi.org/10.1007/978-3-642-00599-2_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00598-5
Online ISBN: 978-3-642-00599-2
eBook Packages: Computer ScienceComputer Science (R0)