With the bombardment of big headlines in the news, our bold statement above sounds preposterous. Technically, though, there are three possible ways that it might actually be true and only one possible way that it is indeed false. With such good odds, we are glad to take our chances.
Under the following three possibilities, our statement is true and thus not preposterous:
- Watson has not beaten a human in Jeopardy and I can’t outrun a Ferrari.
- Watson has not beaten a human in Jeopardy and I can outrun a Ferrari.
- Watson has beaten a human in Jeopardy and I can outrun a Ferrari.
Only under the following one possibility is our statement false and thus preposterous:
- Watson has beaten a human in Jeopardy and I can’t outrun a Ferrari.
Before we examine the possibilities, let’s ask ourselves, “What does it mean for us to say that agent A beats agent B in game C?” It seems clear that minimally we must mean the following:
- By whatever definitions we define agent A and agent B, that is all who can participate in the game, and no one else. The agents can have coaches, trainers or handlers, but they are not allowed in the game.
- The interactions between the contestants A, B and the environment, which includes the host, the referees, the game board and the game pieces etc., are the same for each contestant.
- The rules are pre-fixed, known to each contestant, and apply exactly the same to each.
- If it is a series of games to decide a winner, the same agent must participate in the same games.
Now let’s first examine the easy case, can I outrun a Ferrari? The answer is a resounding “Yes.” First, the definition of “I” and the definition of “a Ferrari” are quite clear. In particular, the definition of “a Ferrari” does not include a driver. After this clarification, everyone must now agree that I can indeed outrun a Ferrari: it does not run at all by itself!
Some will cry foul to our conclusion, by saying, “When we talk about a Ferrari, we obviously mean a Ferrari with its driver.” To this, we have the following answers:
- When you buy a Ferrari, you don’t buy a Ferrari with its driver, so a Ferrari does not obviously mean a Ferrari with its driver.
- It is not a fair fight if I must outrun a Ferrari plus a person as its driver. Would you challenge a person with a gun in a duel without holding a gun yourself?
Now let’s examine if Watson has beaten a human in Jeopardy. The answer is a resounding “No.” Watson doesn’t understand anything that Alex Trebek said! This is confirmed by the fact it does not understand other contestant’s answers. The texts of the questions are electronically fed into Watson instead. Last time I checked, no one who does not understand Alex’s questions has won any Jeopardy games and the texts of the questions are never fed directly into contestants’ brains in any Jeopardy games.
What does it take for any AI to beat human beings in Jeopardy? Minimally, we must first have a clear definition of what the specific AI is. In particular, no humans are allowed in the actual game in any way, shape, or form. And secondly, the AI must understand the questions posed by Alex, the same way all the other contestants do, not directly electronically fed. So, I would say that Watson, on its own, has not yet defeated any humans in Jeopardy and has a long way to go. I am not being preposterous after all.