Abstract
We present a generative model for constructing continuous word representations using mixtures of probabilistic PCAs. Applied to co-occurrence data, the model performs word clustering and allows the visualization of each cluster in a reduced space. In combination with a simple document model, it permits the definition of low-dimensional Fisher scores which are used as document features. We investigate the models’ potential through kernel-based methods using the corresponding Fisher kernels.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Nello Cristianini, Huma Lodhi, and John Shawe-Taylor. Latent semantic kernels for feature selection. Technical Report NC-TR-2000-080, Royal Holloway, University of London, June 2000.
S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 1990.
Thomas Hofmann. Probabilistic latent semantic analysis. Uncertainity in Artificial Intelligence, 1999.
Thomas Hofmann. Learning the similarity of documents: An information-geometric approach to document retrieval and categorization. Advances in Neural Information Processing Systems, 12:914–920, 2000.
Tommi S. Jaakola and David Haussler. Exploiting generative models in discriminative classifiers. In Advances in Neural Information Processing Systems. MIT Press, 1998.
Sabine Ploux and Bernard Victorri. Construction d’espaces semantiques à l’aide de dictionnaires informatisés des synonymes. Traitement Automatique des Langues, 39(1):161–182, 1998.
Sam Scott and Stan Matwin. Text classification using wordnet hypernyms. In Proceedings of the COLING-ACL’98 Workshop, pages 45–52. University of Montreal, Quebec, Canada, 1998.
George Siolas and Florence d’Alché-Buc. Support vector machines based on a semantic kernel for text categorization. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, 2000.
Michael Tipping and Christopher Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society, 61(B):611–622, 1999.
N. Ueda, R. Nakano, Z. Ghahramani, and G. Hinton. Smem algorithm for mixture models. In Advances in Neural Information Processing Systems. MIT Press, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Siolas, G., d’Alché-Buc, F. (2002). Mixtures of Probabilistic PCAs and Fisher Kernels for Word and Document Modeling. In: Dorronsoro, J.R. (eds) Artificial Neural Networks — ICANN 2002. ICANN 2002. Lecture Notes in Computer Science, vol 2415. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46084-5_125
Download citation
DOI: https://doi.org/10.1007/3-540-46084-5_125
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44074-1
Online ISBN: 978-3-540-46084-8
eBook Packages: Springer Book Archive