Abstract
Dimension reduction for a set of distribution parameters has been important in various applications of datamining. The exponential family PCA has been proposed for that purpose, but it cannot be directly applied to mixture models that do not belong to an exponential family. This paper proposes a method to apply the exponential family PCA to mixture models. A key idea is to embed mixtures into a space of an exponential family. The problem is that the embedding is not unique, and the dimensionality of parameter space is not constant when the numbers of mixture components are different. The proposed method finds a sub-optimal solution by linear programming formulation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amari, S.: Differential Geometrical Methods in Statistics. Springer, Heidelberg (1985)
Amari, S.: Information Geometry on Hierarchy of Probability Distributions. IEEE Trans. on Information Theory 41 (2001)
Collins, M., Dasgupta, S., Schapire, R.: A Generalization of Principal Component Analysis to the Exponential Family. In: Advances in NIPS, vol. 14 (2002)
Akaho, S.: The e-PCA and m-PCA: dimension reduction by information geometry. In: IJCNN 2004, pp. 129–134 (2004)
Watanabe, K., Akaho, S., Okada, M.: Clustering on a Subspace of Exponential Family Using Variational Bayes Method. In: Proc. of Worldcomp2008/Information Theory and Statistical Learning (2008)
McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, Chichester (2000)
Agrawal, R., Srikant, R.: Privacy-Preserving Data Mining. In: Proc. of the ACM SIGMOD, pp. 439–450 (2000)
Kumar, A., Kantardzic, M., Madden, S.: Distributed Data Mining: Framework and Implementations. IEEE Internet Computing 10, 15–17 (2006)
Chong, C.Y., Kumar, S.: Sensor networks: evolution, opportunities, and challenges. Proc. of the IEEE 91, 1247–1256 (2003)
Buntine, W.: Variational extensions to EM and multinomial PCA. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430. Springer, Heidelberg (2002)
Edelman, A., Arias, T., Smith, S.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)
Amari, S.: Information geometry of the EM and em algorithms for neural networks. Neural Networks 8(9), 1379–1408 (1995)
Chvátal, V.: Linear Programming. W.H. Freeman and Company, New York (1983)
Fukumizu, K., Akaho, S., Amari, S.: Critical lines in symmetry of mixture models and its application to component splitting. In: Proc. of NIPS15 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Akaho, S. (2008). Dimension Reduction for Mixtures of Exponential Families. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-87536-9_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87535-2
Online ISBN: 978-3-540-87536-9
eBook Packages: Computer ScienceComputer Science (R0)