The first segment of the long-anticipated Jeorpardy tournament between former Jeopardy! champions, Ken Jennings and Brad Rutter, and IBM's supercomputer Watson aired on Feb. 14. Watson and Rutter both had $5,000 on the board and Jennings trailed behind with $2,000.
The first day's final scores reveal only part of the story. Rutter made the first selection, and beat Watson to the buzzer to answer the question. After that, Watson dominated the round up to the commercial break, buzzing in with 11 correct answers out of 15 questions, including the Daily Double (it wagered $1,000 for Literary Character APB). The second half of the round started with $5,200 for Watson, $1,000 for Rutter, and $200 for Jennings on the scoreboard, but Jennings and Rutter beat Watson to the buzzer several times during the course of the round. Most of the wrong answers during this round were also Watson's.
Viewers who expected the computer to get every question right were treated to several of Watson's wrong answers in the second half. Unlike a human player, Watson can't adjust its answers to what other players say and answers whatever it picked as its top answer during initial processing. After Jennings incorrectly answered "20s" was the decade in which Oreo cookies were introduced, Watson answered with "1920s."
Another Watson mistake illustrated the challenges of natural language processing. The category was Olympic Oddities, and the answer was a gymnast with an unusual physical feature. Watson answered "leg," but was ruled incorrect, because the proper answer (editor: no one got right) was that the gymnast's leg was missing.
The categories of the first round were: Literary Characters APB, Beatles People, Olympic Oddities, Name the Decade, Final Frontiers, and Alternate Meanings. The literary characters category was heavy on puns and wordplay, and while Watson was supremely comfortable in the Beatles category, it got every single question in the decades category wrong.
Watson also has an "Achilles heel," because its betting is restricted to its confidence level, according to the book "Final Jeopardy," by technology journalist Stephen Baker.
Only the first round was broadcast, with the remaining Double Jeopardy and Final Jeopardy scheduled for Tuesday. The final game will air on Wednesday. The tournament was taped at the IBM research center in Yorktown Heights, N.Y. in January, but Jeopardy and IBM have managed to prevent the final outcome from leaking. The winner will take home the grand prize of $1 million (and bragging rights) which IBM has said it will donate to charity if Watson wins.
As part of the players' introductions, Trebek walked the TV audience through the server room "next door" to the studio to show the five racks containing ten Power7 servers that house Watson's code, algorithms, and database containing everything it knows. The deep analytics system is the equivalent of 2,800 powerful computers tied together in a super high-speed network, Trebek said. See what else there is to know about Watson.
For more, read the eWeek article IBM's Watson Ties for Lead on Jeopardy! But Makes Some Doozies.