Abstract
In the past decade, Probabilistic Latent Semantic Indexing (PLSI) has become an important modeling technique, widely used in clustering or graph partitioning analysis. However, the original PLSI is designed for multinomial data and may not handle other data types. To overcome this restriction, we generalize PLSI to t-exponential family based on a recently proposed information criterion called t-divergence. The t-divergence enjoys more flexibility than KL-divergence in PLSI such that it can accommodate more types of noise in data. To optimize the generalized learning objective, we propose a Majorization-Minimization algorithm which multiplicatively updates the factorizing matrices. The new method is verified in pairwise clustering tasks. Experimental results on real-world datasets show that PLSI with t-divergence can improve clustering performance in purity for certain datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arora, R., Gupta, M., Kapila, A., Fazel, M.: Clustering by left-stochastic matrix factorization. In: International Conference on Machine Learning (ICML), pp. 761–768 (2011)
Choi, H., Choi, S., Katake, A., Choe, Y.: Learning alpha-integration with partially-labeled data. In: Proc. of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 14–19 (2010)
Cichocki, A., Lee, H., Kim, Y.D., Choi, S.: Non-negative matrix factorization with α-divergence. Pattern Recognition Letters 29, 1433–1440 (2008)
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis. John Wiley (2009)
Ding, C., Li, T., Peng, W.: On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Computational Statistics and Data Analysis 52(8), 3913–3927 (2008)
Ding, C., Li, T., Jordan, M.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1), 45–55 (2010)
Ding, N., Vishwanathan, S.: t-logistic regression. In: Advances in Neural Information Processing Systems, vol. 23, pp. 514–522 (2010)
Ding, N., Vishwanathan, S., Qi, Y.A.: t-divergence based approximate inference. In: Advances in Neural Information Processing Systems, vol. 24, pp. 1494–1502 (2011)
Févotte, C., Bertin, N., Durrieu, J.L.: Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis. Neural Computation 21(3), 793–830 (2009)
Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International Conference on Research and Development in Information Retrieval (SIGIR), pp. 50–57. ACM (1999)
Hunter, D.R., Lange, K.: A tutorial on MM algorithms. The American Statistician 58(1), 30–37 (2004)
Mollah, M., Sultana, N., Minami, M.: Robust extraction of local structures by the minimum of beta-divergence method. Neural Networks 23, 226–238 (2010)
Yang, Z., Oja, E.: Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization. IEEE Transactions on Neural Networks 22(12), 1878–1891 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, H., Hao, T., Yang, Z., Oja, E. (2012). Pairwise Clustering with t-PLSI. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33266-1_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-33266-1_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33265-4
Online ISBN: 978-3-642-33266-1
eBook Packages: Computer ScienceComputer Science (R0)