Imitation Game: Threshold or Watershed?

Abstract

Showing remarkable insight into the relationship between language and thought, Alan Turing in 1950 proposed the Imitation Game as a proxy for the question “Can machines think?” and its meaning and practicality have been debated hotly ever since. The Imitation Game has come under criticism within the Computer Science and Artificial Intelligence communities with leading scientists proposing alternatives, revisions, or even that the Game be abandoned entirely. Yet Turing’s imagined conversational fragments between human and machine are rich with complex instances of inference of implied information, reasoning from generalizations, and meta-reasoning, challenges AI practitioners have wrestled with since at least 1980 and continue to study. We argue that the very fact the Imitation Game is so difficult may be the very reason it shouldn’t be changed or abandoned. The semi-decidability of the game at this point hints at the possibility of a hard limit to the powers of technology.

Appendices

Appendix

Appendix 1: Representing Generalizations with First Order Logic

Summing up the problems encountered during a decade of research necessarily requires oversimplification. We begin with a classic example:

Chilly-Willy is a penguin.

Donald is a duck.

Penguins are birds.

Ducks are birds.

Birds fly.

Penguins don’t fly.

We leave it to the reader to observe that by combining different subsets of this knowledge base, we can show that Chilly-Willy flies, and that Chilly-Willy does not fly. We can eliminate one of these conclusions by designating “birds fly” as a generalization, and adding a rule that an instance of a generalization can only be applied if the set of all sentences used cannot be made to generate a contradiction. We can still use an instance of the generalization to conclude Donald is a bird and can fly. These sentences allow other interesting inferences. For example, taking contrapositive forms, we conclude that if it flies, it’s not a penguin, and if it’s not a bird, it’s not a duck. In this setting, the contrapositive forms make sense, but that isn’t always the case.

Now suppose, ducks are different from most birds in that they have webbed feet. To fully incorporate this into the database, we add the following:

Chilly-Willy is a penguin.

Donald is a duck.

Penguins are birds.

Ducks are birds.

Birds fly.

Penguins don’t fly.

Birds don’t have webbed feet.

Ducks have webbed feet.

Again, this seems reasonable. But suppose we add one more sentence.

The only birds are ducks and penguins.

Although it seems unreasonable to say every bird is either a duck or a penguin, this gives a compact counterexample. Let’s explore the problem with the compact counterexample, then generalize to something more reasonable.

The counterexample goes as follows. Let Foghorn be a bird. Birds typically fly, so if Foghorn flies, Foghorn can’t be a penguin. Because everything is either a penguin or a duck, Foghorn is a duck and has webbed feet. Using the same trick, we can put together a different set of sentences and conclude that Foghorn is a penguin and can’t fly.

Here is a fuller counterexample. Let there be 1000 kinds of birds. Each kind is different from a typical bird in some way, but otherwise is a normal bird. (If the kind has no distinguishing feature from other kinds, how can it be a kind?) Plus, we have the clause that every bird must be one of the 1000 kinds. Using 999 of the generalizations contraposed, we can rule out that Foghorn is not any of those 999 kinds, and therefore must be the remaining kind and be unique in the way the remaining kind is unique.

Some readers will see this as a variation on Kyburg’s lottery paradox, others as a variation on Simpson’s paradox. Either way, the simple reasoning pattern initially proposed has collapsed. This result is our interpretation of (Poole, 1989).

Appendix 2: What Practical Certainty Buys Us

The idea of practical certainty lets us hold as beliefs a set of sentences, which written as a conjunction would contradict some fact. The classic example is a lottery, where the purchaser buys a numbered ticket, and only one number wins, as opposed to modern lotteries where the purchaser can choose their numbers. It is reasonable in such a lottery to believe, for each ticket, that it will lose. But it is not reasonable to believe that no ticket will win, since by construction a winner is drawn. (For an argument that this still holds for the modern lottery, see (Neufeld and Goodwin 1998)).

To keep the calculations simple, let’s suppose there are 20 unique tickets in the lottery. This means that the probability any ticket will lose is 0.95. Thus, it is practically certain that each ticket loses. Suppose an individual buys two tickets. The probability that both tickets lose is 0.9 – this is not a practical certainty, but a probability. In this situation, we cannot combine two practical certainties into a conjunction that is a practical certainty.

As the number of tickets gets large, one can be practically certain that if two tickets are purchased, both will lose.

Applying this to the previous ‘bird’ example, we can’t treat the conjunction that Foghorn is not any one of 999 kinds of birds as a practical certainty. This prevents the collapse of the formalism, but also limits the formalism’s inferential power.

We remark that the example above assumed buying tickets without replacement, which simplified the calculation. More generally, let A and B be any two events of probability 0.95. Thus each is a practical certainty. Using the basic identity.

P(A&B) = P(A) + P(B) – P(A or B).

(where P is probability) the probability of the conjunction could be less than 0.9 because P(A or B) might be 1.

However, if A and B are two arbitrary events of probability 0.99, we can show the lowest value of their conjunction is 0.98 using the same formula, even if the probability of the disjunction is unity, and the conjunction is a practical certainty.

Finally, we remark that the theory of epistemological probability has many nuances. A reader of an earlier draft of this paper asked the following. If 1% of ticks carry Lyme disease, then 99% of ticks do not, and thus it is practically certain that ticks do not carry Lyme disease. This is a knowledge engineering problem worth delving into.

We will use natural language representations of the knowledge rather than introduce a new formalism. To begin with, suppose a data collector has written “Of 300 ticks examined near Gormley Wood, 3 carried Lyme disease”. This might be translated to “The probability of any particular tick carrying Lyme disease is between 0.009 and 0.011”, the interval accounting for all manner of uncertainty about how the data was collected. Next we learn, “Alice Butterwick noticed a tick on her dog Mollie after a walk through Avon Gorge.” From the statistical data, Alice can infer that “the probability the tick on Mollie has Lyme disease is about 1%” (this is lifting) and therefore be practically certain that that tick does not carry the disease. If three hundred dog-owners visit the Gorge every day, it is practically certain someone’s pet will pick up a tick carrying Lyme disease. Similarly, if Alice visits the Gorge with Mollie three hundred times, Mollie is likewise certain to be exposed to a disease bearing tick.

Alice might feel differently about a beloved let getting Lyme disease than about the probability her car will start; in that case she may wish to adjust her level of practical certainty. This also brings in the complications of decision theory. If one thinks in terms of mundane lotteries (where neither positive nor negative outcomes have drastic consequences) rather than diseases, or the commonplace assumptions one makes going about daily business, the conclusions reflect common sense.