Skip to main content

Phase transitions in soft topographic vector quantization

  • Part IV:Signal Processing: Blind Source Separation, Vector Quantization, and Self Organization
  • Conference paper
  • First Online:
Artificial Neural Networks — ICANN'97 (ICANN 1997)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1327))

Included in the following conference series:

Abstract

We have developed an algorithm (STVQ) for the optimization of neighborhood preserving maps by applying deterministic annealing to an energy function for topographic vector quantization. The combinatorial optimization problem is solved by introducing temperature dependent fuzzy assignments of data points to cluster centers and applying an EM-type algorithm at each temperature while annealing. The annealing process exhibits phase transitions in the cluster representation for which we calculate critical modes and temperatures expressed in terms of the neighborhood function and the covariance matrix of the data. In particular, phase transitions corresponding to the automatic selection of feature dimensions are explored analytically and numerically for finite temperatures. Results are related to those obtained earlier for Kohonen's SOM-algorithm which can be derived as an approximation to STVQ. The deterministic annealing approach makes it possible to use the neighborhood function solely to encode desired neighborhood relations. The working of the annealing process is visualized by showing the effects of “heating” on the topological structure of a two-dimensional map of the plane.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. J. M. Buhmann and H. Kühnel. Vector Quantization with Complexity Costs. IEEE Transactions on Information Theory, vol. 39, p. 1133–1145 (1993).

    Google Scholar 

  2. J. M. Buhmann and T. Hofmann. Robust Vector Quantization by Competitive Learning. Proceedings of the International Conference on Accoustics, Speech and Signal Processing ICASSP'97, Munich, (1997).

    Google Scholar 

  3. M. Cottrell, J. C. Fort, and G. Pagès. Two or three things that we know about the Kohonen algorithm. Proceedings of ESANN'94, M. Verleysen Ed., D Facto, Bruxelles, p. 235–244 (1995).

    Google Scholar 

  4. E. Erwin, K. Obermayer, and K. Schulten. Self-Organizing Maps: Ordering, Convergence Properties and Energy Functions. Biological Cybernetics, vol. 67, p. 47–55 (1992).

    Google Scholar 

  5. T. Kohonen. Self-Organizing Maps. Springer-Verlag, 1995.

    Google Scholar 

  6. S. P. Luttrell. Self-Organisation: A Derivation from first Principles of a Class of Learning Algorithms. IEEE International Joint Conference on Neural Networks: IJCNN, Washington DC, vol. 2, p. 495–498 (1989).

    Google Scholar 

  7. S. P. Luttrell. A Baysian Analysis of Self-Organizing Maps. Neural Computation, vol. 6, p. 767–794 (1994).

    Google Scholar 

  8. K. Obermayer, G. G. Blasdel, and K. Schulten. Statistical-Mechanical Analysis of Self-Organization and Pattern Formation during the Development of Visual Maps. Physical Review A, vol. 45, p. 7568–7589 (1992).

    Google Scholar 

  9. H. Ritter and K. Schulten. Convergence Properties of Kohonen's Topology Conserving Maps: Fluctuations, Stability, and Dimension Selection. Biological Cybernetics, vol. 60, p. 59–71 (1988).

    Google Scholar 

  10. K. Rose, E. Gurewitz, and G. C. Fox. Statistical Mechanics and Phase Transitions in Clustering. Physical Review Letters, vol. 65, p. 945–948 (1990).

    Google Scholar 

  11. Y. Wong. Clustering Data by Melting. Neural Computation, vol. 5, p. 89–104 (1993).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Wulfram Gerstner Alain Germond Martin Hasler Jean-Daniel Nicoud

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Burger, M., Graepel, T., Obermayer, K. (1997). Phase transitions in soft topographic vector quantization. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020223

Download citation

  • DOI: https://doi.org/10.1007/BFb0020223

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-63631-1

  • Online ISBN: 978-3-540-69620-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics