Loading [a11y]/accessibility-menu.js
Finding needles in the haystack: Search and candidate generation | IBM Journals & Magazine | IEEE Xplore

Finding needles in the haystack: Search and candidate generation


Abstract:

A key phase in the DeepQA architecture is Hypothesis Generation, in which candidate system responses are generated for downstream scoring and ranking. In the IBM Watson™ ...Show More

Abstract:

A key phase in the DeepQA architecture is Hypothesis Generation, in which candidate system responses are generated for downstream scoring and ranking. In the IBM Watson™ system, these hypotheses are potential answers to Jeopardy!™ questions and are generated by two components: search and candidate generation. The search component retrieves content relevant to a given question from Watson's knowledge resources. The candidate generation component identifies potential answers to the question from the retrieved content. In this paper, we present strategies developed to use characteristics of Watson's different knowledge sources and to formulate effective search queries against those sources. We further discuss a suite of candidate generation strategies that use various kinds of metadata, such as document titles or anchor texts in hyperlinked documents. We demonstrate that a combination of these strategies brings the correct answer into the candidate answer pool for 87.17% of all the questions in a blind test set, facilitating high end-to-end question-answering performance.
Published in: IBM Journal of Research and Development ( Volume: 56, Issue: 3.4, May-June 2012)
Page(s): 6:1 - 6:12

ISSN Information:


References

References is not available for this document.