Abstract
We introduce a well-grounded minimum description length (MDL) based quality measure for a clustering consisting of either spherical or axis-aligned normally distributed clusters and a cluster with a uniform distribution in an axis-aligned rectangular box. The uniform component extends the practical usability of the model e.g. in the presence of noise, and using the MDL principle for the model selection makes comparing the quality of clusterings with a different number of clusters possible. We also introduce a novel search heuristic for finding the best clustering with an unknown number of clusters. The heuristic is based on the idea of moving points from the Gaussian clusters to the uniform one and using MDL for determining the optimal amount of noise. Tests with synthetic data having a clear cluster structure imply that the search method is effective in finding the intuitively correct clustering.
Supported by Academy of Finland grant 118653 (Algodan) and the PASCAL Network of Excellence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: SODA 2007: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete algorithms, Philadelphia, PA, USA, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1–38 (1977)
Grünwald, P.D.: The Minimum Description Length Principle. The MIT Press, Cambridge (2007)
Grünwald, P.D., Myung, I.J., Pitt, M.A. (eds.): Advances in Minimum Description Length Theory and Applications. The MIT Press, Cambridge (2005)
Kontkanen, P.: Computationally Efficient Methods for MDL-Optimal Density Estimation and Data Clustering. PhD thesis, University of Helsinki, Department of Computer Science (2009)
Meilă, M.: Comparing clusterings–an information based distance. Journal of Multivariate Analysis 98(5), 873–895 (2007)
Rissanen, J.: Modeling by shortest data description. Automatica 14(5), 465–471 (1978)
Rissanen, J.: A universal prior for integers and estimation by minimum description length. The Annals of Statistics 11(2), 416–431 (1983)
Rissanen, J.: Stochastic complexity. Journal of the Royal Statistical Society. Series B (Methodological) 49(3), 223–239 (1987)
Rissanen, J.: Fisher information and stochastic complexity. IEEE Transactions on Information Theory 42(1), 40–47 (1996)
Rissanen, J.: Information and Complexity in Statistical Modeling. Springer, New York (2007)
Szpankowski, W.: Average case analysis of algorithms on sequences. John Wiley & Sons, Chichester (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Luosto, P., Kivinen, J., Mannila, H. (2010). Gaussian Clusters and Noise: An Approach Based on the Minimum Description Length Principle. In: Pfahringer, B., Holmes, G., Hoffmann, A. (eds) Discovery Science. DS 2010. Lecture Notes in Computer Science(), vol 6332. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16184-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-16184-1_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16183-4
Online ISBN: 978-3-642-16184-1
eBook Packages: Computer ScienceComputer Science (R0)