Abstract
The elastic net and related algorithms, such as generative topographic mapping, are key methods for discretized dimension-reduction problems. At their heart are priors that specify the expected topological and geometric properties of the maps. However, up to now, only a very small subset of possible priors has been considered. Here we study a much more general family originating from discrete, high-order derivative operators. We show theoretically that the form of the discrete approximation to the derivative used has a crucial influence on the resulting map. Using a new and more powerful iterative elastic net algorithm, we confirm these results empirically, and illustrate how different priors affect the form of simulated ocular dominance columns.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Durbin, R., Willshaw, D.: An analogue approach to the traveling salesman problem using an elastic net method. Nature 326, 689–691 (1987)
Durbin, R., Szeliski, R., Yuille, A.: An analysis of the elastic net approach to the traveling salesman problem. Neural Computation 1, 348–358 (1989)
Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Computation 10, 215–234 (1998)
Kohonen, T.K.: Self-Organizing Maps. Springer, Heidelberg (1995)
Durbin, R., Mitchison, G.: A dimension reduction framework for understanding cortical maps. Nature 343, 644–647 (1990)
Goodhill, G.J., Willshaw, D.J.: Application of the elastic net algorithm to the formation of ocular dominance stripes. Network: Computation in Neural Systems 1, 41–59 (1990)
Dayan, P.: Arbitrary elastic topologies and ocular dominance. Neural Computation 5, 392–401 (1993)
Carreira-Perpiñán, M.Á.: Continuous Latent Variable Models for Dimensionality Reduction and Sequential Data Reconstruction. PhD thesis, University of Sheffield, UK (2001)
Carreira-Perpiñán, M.Á., Goodhill, G.J.: Generalized elastic nets (2005) (in revision)
Golub, G.H., van Loan, C.F.: Matrix Computations. Johns Hopkins University Press, Baltimore (1996)
Carreira-Perpiñán, M.Á., Goodhill, G.J.: Influence of lateral connections on the structure of cortical maps. J. Neurophysiol. 92, 2947–2959 (2004)
Utsugi, A.: Hyperparameter selection for self-organizing maps. Neural Computation 9, 623–635 (1997)
Samarskiĭ, A.A.: The Theory of Difference Schemes. Marcel Dekker, New York (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Carreira-Perpiñán, M.Á., Dayan, P., Goodhill, G.J. (2005). Differential Priors for Elastic Nets. In: Gallagher, M., Hogan, J.P., Maire, F. (eds) Intelligent Data Engineering and Automated Learning - IDEAL 2005. IDEAL 2005. Lecture Notes in Computer Science, vol 3578. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11508069_44
Download citation
DOI: https://doi.org/10.1007/11508069_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26972-4
Online ISBN: 978-3-540-31693-0
eBook Packages: Computer ScienceComputer Science (R0)