Abstract
This work presents a mathematical treatment of the relation between Self-Organizing Maps (SOMs) and Gaussian Mixture Models (GMMs). We show that energy-based SOM models can be interpreted as performing gradient descent, minimizing an approximation to the GMM log-likelihood that is particularly valid for high data dimensionalities. The SOM-like decrease of the neighborhood radius can be understood as an annealing procedure ensuring that gradient descent does not get stuck in undesirable local minima. This link allows to treat SOMs as generative probabilistic models, giving a formal justification for using SOMs, e.g., to detect outliers, or for sampling.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cottrell, M., Fort, J.C., Pagès, G.: Two or three things that we know about the Kohonen algorithm. In: Proceedings of ESANN 1994, pp. 235–244 (1994)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data Via the EM Algorithm, vol. 39. Wiley Online Library (1977). https://doi.org/10.1111/j.2517-6161.1977.tb01600.x
Dognin, P.L., Goel, V., Hershey, J.R., Olsen, P.A.: A fast, accurate approximation to log likelihood of Gaussian mixture models. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, vol. 3, pp. 3817–3820 (2009). https://doi.org/10.1109/ICASSP.2009.4960459
Gepperth, A.: An energy-based SOM model not requiring periodic boundary conditions. Neural Comput. Appl. (2019). https://doi.org/10.1007/s00521-019-04028-9
Hartley, H.O.: Maximum likelihood estimation from incomplete data. Biometrics 14, 174–194 (1958)
Hecht, T., Lefort, M., Gepperth, A.: Using self-organizing maps for regression: the importance of the output function. In: 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN - Proceedings, pp. 107–112 (2015)
Heskes, T.: Energy functions for self-organizing maps. Kohonen Maps 303–315 (1999). https://doi.org/10.1016/b978-044450270-4/50024-3
Heskes, T.: Self-organizing maps, vector quantization, and mixture modeling. IEEE Trans. Neural Netw. 12(6), 1299–1305 (2001). https://doi.org/10.1109/72.963766
Hosseini, R., Sra, S.: Matrix manifold optimization for Gaussian mixtures. In: Advances in Neural Information Processing Systems, pp. 910–918 (2015)
Kohonen, T.: The self-organizing map. Proc. IEEE 78(9), 1464–1480 (1990)
Ormoneit, D., Tresp, V.: Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates. IEEE Trans. Neural Netw. 9(4), 639–650 (1998). https://doi.org/10.1109/72.701177
Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22(3), 400–407 (1951). https://doi.org/10.1214/aoms/1177729586
Sherif, A., Ezzat, E.S.: Ahdbase. http://datacenter.aucegypt.edu/shazeem/
Verbeek, J.J., Vlassis, N., Kröse, B.J.: Self-organizing mixture models. Neurocomputing 63, 99–123 (2005). https://doi.org/10.1016/j.neucom.2004.04.008
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gepperth, A., Pfülb, B. (2020). A Rigorous Link Between Self-Organizing Maps and Gaussian Mixture Models. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12397. Springer, Cham. https://doi.org/10.1007/978-3-030-61616-8_69
Download citation
DOI: https://doi.org/10.1007/978-3-030-61616-8_69
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61615-1
Online ISBN: 978-3-030-61616-8
eBook Packages: Computer ScienceComputer Science (R0)