Abstract
Spectral clustering can provide surprising performances. As all kernel methods, is uses a similarity matrix, whose size grows with n 2, and it requires to solve a possibly large eigenproblem. In this paper we focus on a method for spectral embedding of stream data, modeled as an unbounded quantity of input observation. A second purpose of this work is to analyze the proposed method and compare it with traditional neural network implementations: current knowledge about computations in neurons and the brain does not contrast with the computing primitives required for a local implementation of the proposed technique. A hypothesis stemming from this work could be that concept formation and discrimination in neurons and the brain could be explained by a spectral embedding framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alger, B., Pitler, T.: Retrograde signaling at GABAA-receptor synapses in the mammalian CNS. Trends in Neurosciences 18(8), 333–340 (1995)
Anderson, E.: The irises of the gaspe peninsula. Bulletin of the American Iris Society 59, 25 (1935)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)
Bengio, Y., Paiement, J.F., Vincent, P., Delalleau, O., Le Roux, N., Ouimet, M.: Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering. Mij 1, 2 (2003)
Carandini, M., Heeger, D.J.: Normalization as a canonical neural computation. Nature Reviews Neuroscience 13(1), 51–62 (2012)
Chung, F.R.K.: Spectral Graph Theory (CBMS Regional Conference Series in Mathematics, No. 92). American Mathematical Society (February 1997)
De Silva, V., Tenenbaum, J.B.: Sparse multidimensional scaling using landmark points. Tech. rep., Technical report, Stanford University (2004)
Drineas, P., Mahoney, M.W.: On the nyström method for approximating a gram matrix for improved kernel-based learning. The Journal of Machine Learning Research 6, 2153–2175 (2005)
Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A survey of kernel and spectral methods for clustering. Pattern Recognition 40(1), 176–190 (2008)
Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the nystrom method. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(2), 214–225 (2004)
Heeger, D.J.: Half-squaring in responses of cat striate cells. Visual Neuroscience 9, 427–443 (1992)
Homa, D., Cornell, D., Goldman, D., Shwartz, S.: Prototype abstraction and classification of new instances as a function of number of instances defining the prototype. Journal of Experimental Psychology 101(1), 116 (1973)
Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology 160(1), 106 (1962)
Hubel, D.H., Wiesel, T.N.: Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. The Journal of Neurophysiology 28(2), 229 (1965)
Nadler, B., Galun, M.: Fundamental limitations of spectral clustering. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems 19, pp. 1017–1024. MIT Press, Cambridge (2007)
Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14. MIT Press, Cambridge (2002)
Ning, H., Xu, W., Chi, Y., Gong, Y., Huang, T.S.: Incremental spectral clustering by efficiently updating the eigen-system. Pattern Recognition 43(1), 113–127 (2010)
Oja, E.: Neural networks, principal components, and subspaces. International Journal of Neural Systems 01(01), 61–68 (1989)
Oja, E., Karhunen, J.: On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix. Journal of Mathematical Analysis and Applications 106(1), 69–84 (1985)
Quiroga, R.Q., Reddy, L., Kreiman, G., Koch, C., Fried, I.: Invariant visual representation by single neurons in the human brain. Nature 435(7045), 1102–1107 (2005)
Quiroga, R.Q.: Concept cells: the building blocks of declarative memory functions. Nature Reviews Neuroscience 13(8), 587–597 (2012)
Ridella, S., Rovetta, S., Zunino, R.: Circular back–propagation networks for classification. IEEE Transactions on Neural Networks 8(1), 84–97 (1997)
Rovetta, S., Zunino, R.: Circular backpropagation networks embed vector quantization. IEEE Transactions on Neural Networks 10(4), 972–975 (1999)
Sanger, T.D.: Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2(6), 459–473 (1989)
Stuart, G., Spruston, N., Sakmann, B., Häusser, M.: Action potential initiation and backpropagation in neurons of the mammalian CNS. Trends in Neurosciences 20(3), 125–131 (1997)
Von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17(4), 395–416 (2007)
Yan, D., Huang, L., Jordan, M.I.: Fast approximate spectral clustering. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 907–916. ACM (2009)
Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems, pp. 1601–1608 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Rovetta, S., Masulli, F. (2015). Online Spectral Clustering and the Neural Mechanisms of Concept Formation. In: Bassis, S., Esposito, A., Morabito, F. (eds) Advances in Neural Networks: Computational and Theoretical Issues. Smart Innovation, Systems and Technologies, vol 37. Springer, Cham. https://doi.org/10.1007/978-3-319-18164-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-18164-6_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18163-9
Online ISBN: 978-3-319-18164-6
eBook Packages: EngineeringEngineering (R0)