Abstract
General finite mixture models are powerful tools for the density-based grouping of multivariate i.i.d. data, but they lack data visualization capabilities, which reduces their practical applicability to real-world problems. Generative topographic mapping (GTM) was originally formulated as a constrained mixture of distributions in order to provide simultaneous visualization and clustering of multivariate data. In its inception, the adaptive parameters were determined by maximum likelihood (ML), using the expectation-maximization (EM) algorithm. The original GTM is, therefore, prone to data overfitting unless a regularization mechanism is included. In this paper, we define an alternative variational formulation of GTM that provides a full Bayesian treatment to a Gaussian process (GP)-based variation of the model. The generalization capabilities of the proposed Variational Bayesian GTM are assessed in some detail and compared with those of alternative GTM regularization approaches in terms of test log-likelihood, using several artificial and real datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abrahamsen, P.: A review of Gaussian random fields and correlation functions. Tech. Rep. 917, Norwegian Computing Center, Oslo, Norway (1997)
Andrieu, C., de Freitas, N., Doucet, A., Jordan, M.I.: An introduction to MCMC for machine learning. Mach. Learn. 50, 5–43 (2003)
Baum, L., Egon, J.: An inequality with applications to statistical estimation for probabilistic functions for a Markov process and to a model for ecology. B. Am. Meteorol. Soc. 73, 360–363 (1967)
Beal, M.: Variational algorithms for approximate Bayesian inference. Ph.D. thesis, The Gatsby Computational Neuroscience Unit, Univ. College London (2003)
Bishop, C.M.: Variational principal components. In: Proceedings Ninth Intern. Conf. on Artificial Neural Networks, vol. 1, pp. 509–514 (1999)
Bishop, C.M., Svensén, M., Williams, C.K.I.: Developments of the generative topographic mapping. Neurocomputing 21(1–3), 203–224 (1998)
Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Comput. 10(1), 215–234 (1998)
Gibbs, M., MacKay, D.J.C.: Variational Gaussian process classifiers. IEEE Trans. Neural Netw. 11(6), 1458–1464 (2000)
Jakkola, T., Jordan, M.I.: Bayesian parameter estimation via variational methods. Stat. Comput. 10, 25–33 (2000)
Kohonen, T.: Self-Organizing Maps (3rd ed). Springer-Verlag, Berlin (2001)
MacKay, D.J.C.: Bayesian interpolation. Neural Comput. 4(3), 415–447 (1992)
MacKay, D.J.C.: A practical Bayesian framework for back-propagation networks. Neural Comput. 4(3), 448–472 (1992)
Utsugi, A.: Bayesian sampling and ensemble learning in generative topographic mapping. Neural Process. Lett. 12, 277–290 (2000)
Vellido, A.: Missing data imputation through GTM as a mixture of t-distributions. Neural Netw. 19(10), 1624–1635 (2006)
Vellido, A., El-Deredy, W., Lisboa, P.J.G.: Selective smoothing of the generative topographic mapping. IEEE Trans. Neural Netw. 14(4), 847–852 (2003)
Vellido, A., Lisboa, P.J.G.: Handling outliers in brain tumour MRS data analysis through robust topographic mapping. Comput. Biol. Med. 3(10), 1049–1063 (2006)
Vellido, A., Lisboa, P.J.G., Vicente, D.: Robust analysis of MRS brain tumour data using t-GTM. Neurocomputing 69(7–9), 754–768 (2006)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Olier, I., Vellido, A. Variational Bayesian Generative Topographic Mapping. J Math Model Algor 7, 371–387 (2008). https://doi.org/10.1007/s10852-008-9088-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10852-008-9088-7