Generalizing Self-organizing Maps: Large-Scale Training of GMMs and Applications in Data Science

Abstract

This contribution shows that Gaussian Mixture Models can be considered generalizations of self-organizing maps. More precisely, we demonstrate that the training of self-organizing maps is an approximation to the training of Gaussian Mixture Models by gradient descent. As a consequence, the scores of a trained SOM can be treated as log-likelihoods of a GMM with tied, spherical covariance and used, e.g., for outlier detection, whereas sampling from trained SOMs is not well-defined. Furthermore, we outline how SGD-trained GMMs can be generalized to diagonal and more expressive covariance matrices and how this benefits typical data science applications such as outlier detection, sampling and generative classification. Source codes are available on the author’s web site or upon request.