Abstract
We have developed an algorithm (STVQ) for the optimization of neighborhood preserving maps by applying deterministic annealing to an energy function for topographic vector quantization. The combinatorial optimization problem is solved by introducing temperature dependent fuzzy assignments of data points to cluster centers and applying an EM-type algorithm at each temperature while annealing. The annealing process exhibits phase transitions in the cluster representation for which we calculate critical modes and temperatures expressed in terms of the neighborhood function and the covariance matrix of the data. In particular, phase transitions corresponding to the automatic selection of feature dimensions are explored analytically and numerically for finite temperatures. Results are related to those obtained earlier for Kohonen's SOM-algorithm which can be derived as an approximation to STVQ. The deterministic annealing approach makes it possible to use the neighborhood function solely to encode desired neighborhood relations. The working of the annealing process is visualized by showing the effects of “heating” on the topological structure of a two-dimensional map of the plane.
Preview
Unable to display preview. Download preview PDF.
References
J. M. Buhmann and H. Kühnel. Vector Quantization with Complexity Costs. IEEE Transactions on Information Theory, vol. 39, p. 1133–1145 (1993).
J. M. Buhmann and T. Hofmann. Robust Vector Quantization by Competitive Learning. Proceedings of the International Conference on Accoustics, Speech and Signal Processing ICASSP'97, Munich, (1997).
M. Cottrell, J. C. Fort, and G. Pagès. Two or three things that we know about the Kohonen algorithm. Proceedings of ESANN'94, M. Verleysen Ed., D Facto, Bruxelles, p. 235–244 (1995).
E. Erwin, K. Obermayer, and K. Schulten. Self-Organizing Maps: Ordering, Convergence Properties and Energy Functions. Biological Cybernetics, vol. 67, p. 47–55 (1992).
T. Kohonen. Self-Organizing Maps. Springer-Verlag, 1995.
S. P. Luttrell. Self-Organisation: A Derivation from first Principles of a Class of Learning Algorithms. IEEE International Joint Conference on Neural Networks: IJCNN, Washington DC, vol. 2, p. 495–498 (1989).
S. P. Luttrell. A Baysian Analysis of Self-Organizing Maps. Neural Computation, vol. 6, p. 767–794 (1994).
K. Obermayer, G. G. Blasdel, and K. Schulten. Statistical-Mechanical Analysis of Self-Organization and Pattern Formation during the Development of Visual Maps. Physical Review A, vol. 45, p. 7568–7589 (1992).
H. Ritter and K. Schulten. Convergence Properties of Kohonen's Topology Conserving Maps: Fluctuations, Stability, and Dimension Selection. Biological Cybernetics, vol. 60, p. 59–71 (1988).
K. Rose, E. Gurewitz, and G. C. Fox. Statistical Mechanics and Phase Transitions in Clustering. Physical Review Letters, vol. 65, p. 945–948 (1990).
Y. Wong. Clustering Data by Melting. Neural Computation, vol. 5, p. 89–104 (1993).
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Burger, M., Graepel, T., Obermayer, K. (1997). Phase transitions in soft topographic vector quantization. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020223
Download citation
DOI: https://doi.org/10.1007/BFb0020223
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63631-1
Online ISBN: 978-3-540-69620-9
eBook Packages: Springer Book Archive