Who wants a computer to be a millionaire?

https://doi.org/10.1016/j.ipl.2014.12.005Get rights and content

Highlights

  • •

    We focus on “Who wants to be a Millionaire?” show as multiple-choice question answering.

  • •

    The search snippet frequency model that works with a search engine is proposed.

  • •

    DBpedia, as a knowledge base, is further proposed for the task.

  • •

    Combining DBpedia with the search snippet frequency method shows the best performance.

  • •

    Our system outperforms human contestants and becomes millionaire in 6 out of 50 runs.

Abstract

Competing against computers was one of the important challenges in the last decades. People like to compare their abilities with computers that are, in fact, their own invention. TV game shows provide a good opportunity for such competitions. Several attempts have been made to find out how sophisticated systems fare in game shows. An example of the task is competing in games with multiple-choice questions, such as “Who wants to be a Millionaire”.

We propose an approach to this problem by using search engines and knowledge bases to automatically select the answer. The experimental results indicate the superiority of the proposed model over related work. Our proposed method achieved average winnings of $250,000 on a US question set and became a millionaire six times, out of fifty runs, which is much higher than the normal winning rate among human contestants.

Section snippets

Computers and games

Games can be broadly categorized into the classes athletic, strategic, and knowledge-based games (and combinations). Human contestants in all three classes are under more and more pressure from machines: robots are increasingly adept in athletics, for instance the RoboCup soccer championships. In strategic games, such as chess or checkers, computers have made much more headway, for instance IBM's Deep Blue computer won over world champion Garri Kasparov in 1997 [3]. As more and more knowledge

Scoring strategies

There are various techniques to select an answer using information retrieval and knowledge processing methods. We first describe several approaches from the literature, and then propose two novel approaches for this task, namely search snippet frequency and DBpedia spotlight.

For all techniques used in previous studies and also our proposed search snippet frequency approach, we need to gather web data using a search engine. To this aim, we use the retrieval API of Google.

Experimental results

To evaluate our system, we used a set of 2027 questions from the TV game show “Who Wants to be a Millionaire” out of which 1012 questions are from the UK and 1015 questions are from the US version of the show. The dataset is provided from a Game Boy Advance ROM file.

Table 1 shows the accuracy (percentage of correctly answered questions) of different strategies including previous and our proposed approaches.

Comparing the result of the UK vs. US questions, we observe that most of the strategies

Concluding remarks

We presented two new strategies to answer multiple-choice questions. The first strategy, which uses a search engine as information source, tries to find the best answer without any bias toward answer choices by avoiding the answer terms in the query and searching for them only in the search snippets. The second strategy employs a new resource, DBpedia, for finding the correct answer. Although this knowledge-base strategy is not able to outperform other strategies individually, it improves the

Acknowledgements

The authors thank the students of the ‘Question Answering’ seminar at HPI, Cindy Fähnrich, Marian Gawron, Stefan Klauck, Matthias Kohnen, Sebastian Kölle, Philipp Langer, Minh Tuan Nguyen, Sebastian Oergel, Nils Rethmeier, Oliver Richter, Patrick Schulze, and Gary Yao for their contributions in developing ideas and implementing the system.

References (6)

There are more references available in the full text version of this article.

Cited by (0)

1

Present address: Ubiquitous Knowledge Processing Lab (UKP-DIPF), German Institute for Educational Research and Educational Information, Schlossstrasse 29, 60486 Frankfurt/Main, Germany. Tel.: +49 69 24708866.

View full text
Copyright © 2014 Elsevier B.V. All rights reserved.