Abstract
Probabilistic topic models have become a standard in modern machine learning with wide applications in organizing and summarizing ‘documents’ in high-dimensional data such as images, videos, texts, gene expression data, and so on. Representing data by dimensional reduction of mixture proportion extracted from topic models is not only richer in semantics than bag-of-word interpretation, but also more informative for classification tasks. This paper describes the Topic Model Kernel (TMK), a high dimensional mapping for Support Vector Machine classification of data generated from probabilistic topic models. The applicability of our proposed kernel is demonstrated in several classification tasks from real world datasets. We outperform existing kernels on the distributional features and give the comparative results on non-probabilistic data types.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blei, D., Ng, A., Jordan, M.: Latent Dirichlet allocation. Journal of Machine Learning Research 3, 993–1022 (2003)
Teh, Y., Jordan, M., Beal, M., Blei, D.: Hierarchical Dirichlet processes. Journal of the American Statistical Association 101, 1566–1581 (2006)
Fritz, M., Schiele, B.: Decomposition, discovery and detection of visual categories using topic models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)
Antolín, J., Angulo, J., López-Rosa, S.: Fisher and jensen–shannon divergences: Quantitative comparisons among distributions application to position and momentum atomic densities. The Journal of Chemical Physics 130, 074110 (2009)
Nguyen, T., Phung, D., Gupta, S., Venkatesh, S.: Extraction of latent patterns and contexts from social honest signals using hierarchical dirichlet processes. In: IEEE International Conference on Pervasive Computing and Communications, PerCom 2013 (2013)
Endres, D., Schindelin, J.: A new metric for probability distributions. IEEE Transactions on Information Theory 49, 1858–1860 (2003)
Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers, 144–152 (1992)
Chang, C., Lin, C.: Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2, 27 (2011)
Nguyen, T.V., Phung, D., Venkatesh, S.: Topic model kernel: An empirical study towards probabilistically reduced features for classification. Technical report, Pattern Recognition and Data Analytics, Deakin University (2013)
Kullback, S., Leibler, R.: On information and sufficiency. The Annals of Mathematical Statistics 22, 79–86 (1951)
Moreno, P.J., Ho, P., Vasconcelos, N.: A kullback-leibler divergence based kernel for svm classification in multimedia applications. Advances in Neural Information Processing Systems 16, 1385–1393 (2003)
Chan, A.B., Vasconcelos, N., Moreno, P.J.: A family of probabilistic kernels based on information divergence. Univ. California, San Diego, CA, Tech. Rep. SVCL-TR-2004-1 (2004)
Topsoe, F.: Jenson-shannon divergence and norm-based measures of discrimination and variation (2003) (preprint)
Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42, 145–175 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nguyen, TV., Phung, D., Venkatesh, S. (2013). Topic Model Kernel: An Empirical Study towards Probabilistically Reduced Features for Classification. In: Lee, M., Hirose, A., Hou, ZG., Kil, R.M. (eds) Neural Information Processing. ICONIP 2013. Lecture Notes in Computer Science, vol 8227. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42042-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-42042-9_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-42041-2
Online ISBN: 978-3-642-42042-9
eBook Packages: Computer ScienceComputer Science (R0)