Abstract
Topic models are hierarchical probabilistic models for the statistical analysis of document collections. It assumes that each document comprises a mixture of latent topics and each topic can be represented by a distribution over vocabulary. Dimensionality for a large corpus of unstructured documents can be reduced by modeling with these exchangeable topics. In previous work, we designed a multi-pipe structure for question answering (QA) systems by nesting keyword search, classical Natural Language Processing (NLP) techniques and prototype detections. In this research, we use those technologies to select a set of sentences as candidate answers. We then use topic models to rank these candidate answers by calculating the semantic distances between these sentences and the given query. In our experiments, we found that the new model of using topic models improves the answer ranking so that the better answers can returned for the given query.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blei, D.M., Griffiths, T., Jordan, M.I., Tenenbaum, J.: Hierarchical Topic Models and the Nested Chinese Restaurant Process. In: Thrun, S., Saul, L., Schoelkopf, B. (eds.) Advances in Neural Information Processing Systems (2004)
Blei, D.M., Lafferty, J.D.: Dynamic Topic Model. In: Proceedings of the 23rd ICML, Pittsburgh, USA (2006)
Blei, D.M., Lafferty, J.D.: Correlated Topic Models. In: Advances in Neural Information Processing Systems 18. MIT Press, Cambridge (2006)
Blei, D.M., Ng, A., Jordan, M.I.: Latent Dirichlet Allocation. Journal of Machine Leanring Research 3, 993–1022 (2003)
Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by Latent Semantic Analysis. Journal of the American Society of Information Science 41 (1990)
Gospodnetic, O., Hatcher, E.: Lucene in Action, Manning (2004)
Griffiths, T.L., Steyvers, M.: Finding Scientific Topics. Proceedings of the National Academy of Science 101, 5228–5235 (2004)
Miller, G.: WordNet: a lexical database. Communications of the ACM 38(11), 39–41 (1995)
Hofmann, T.: Probabilistic Latent Semantic Anlalysis. In: Proceedings of UAI 1999, Stockholm (1999)
Jordan, M.I.: Learning in Graphical Models. MIT Press, Cambridge (1999)
Beg, M.M.S., Thint, M., Qin, Z.: PNL-enhanced Restricted Domain Question Answering System. In: The Proceedings of IEEE-FUZZ, London, pp. 1277–1283. IEEE Press, Los Alamitos (1999)
Qin, Z., Thint, M., Beg, M.M.S.: Deduction Engine Designs for PNL-based Question Answering Systems. In: Melin, P., Castillo, O., Aguilar, L.T., Kacprzyk, J., Pedrycz, W. (eds.) IFSA 2007. LNCS (LNAI), vol. 4529, pp. 253–262. Springer, Heidelberg (2007)
Steyvers, M., Griffiths, T.: Probabilistic topic models. In: Landauer, T., McNamara, D., Dennis, S., Kintsch, W. (eds.) Latent Semantic Analysis’ A Road to Meaning (2007)
Wei, X., Croft, W.B.: LDA-based document models for ad-hoc retrieval. In: SIGIR 2006, Seattle, WA (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Qin, Z., Thint, M., Huang, Z. (2009). Ranking Answers by Hierarchical Topic Models. In: Chien, BC., Hong, TP., Chen, SM., Ali, M. (eds) Next-Generation Applied Intelligence. IEA/AIE 2009. Lecture Notes in Computer Science(), vol 5579. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02568-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-02568-6_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02567-9
Online ISBN: 978-3-642-02568-6
eBook Packages: Computer ScienceComputer Science (R0)