IBM Takes on Jeopardy.

A place for all tech stuff that doesn't really belong anywhere else.
Post Reply
Anakha56
Forum Administrator
Posts: 22136
Joined: 14 Jun 2004, 02:00
Processor: Ryzen 1700K
Motherboard: Asus X370
Graphics card: Asus 1060 Strix
Memory: 16GB RAM
Location: Where Google says

IBM Takes on Jeopardy.

Post by Anakha56 »

I am surprised that this has not made news on PCF, Drak must be sleeping :P.

For those of you who dont know IBM decided that the next step in their computer vs human challenge would be Jeopardy. And so far the computer is winning by a staggering amount of points:

http://www.informationweek.com/news/har ... =229218771
IBM's Watson Puts Human Rivals In Jeopardy
Big Blue's supercomputer wins both Daily Doubles, but thinks Toronto is a U.S. city.

By Paul McDougall , InformationWeek
February 16, 2011 12:30 PM

IBM's Watson supercomputer won the first round of a two-match Jeopardy standoff after crushing his carbon-based competitors Tuesday.

Watson ended the first match, which aired over two nights, with $35,734 in winnings. All-time Jeopardy champ Brad Rutter was second, with $10,400 while Ken Jennings, who holds the record for the game show's longest winning streak, was third with $4,800.

In racking up its total, Watson answered 24 of 30 questions correctly and won both Daily Doubles. But the computer also showed that even the most advanced artificial intelligence program can be strikingly fallible at times. It drew guffaws from the audience when it answered "Toronto" to a question that asked which city was home to airports named after a World War II hero and a famous WWII battle.

Watson, in his placid mechanical voice, named the Canadian city despite the fact the question was under the category "U.S. Cities."

On its Smarter Planet blog, IBM said Watson's training may have led to the error. "Watson, in his training phase, learned that categories only weakly suggest the kind of answer that is expected, and therefore the machine downgrades their significance," IBM said.

Watson, named after IBM founder Thomas J. Watson, is much more than a science experiment. Much of the program is built on technology that IBM has already commercialized for applications such as economic modeling, weather forecasting, the prediction of disease vectors, and the tracking of trends in financial markets.

"Beyond our excitement for the match itself, our team is very motivated by the possibilities that Watson's breakthrough computing capabilities hold for building a smarter planet and helping people in their business tasks and personal lives," said David Ferrucci, who leads IBM's Watson team.

Watson simultaneously runs natural language processing, information retrieval, knowledge representation and reasoning algorithms to fathom the intent of questions and yield what it thinks is the best answer—all in a matter of seconds or less. The Watson program runs on IBM's new massively parallel POWER7 processors, which the company rolled out last year.

Watson is competing against Rutter and Jennings for a grand prize of $1 million. The two human contestants have pledged to give 50% of their winnings to charity if they prevail, while IBM will donate 100% to charity if Watson wins.

The contest resumes Wednesday.
Another write up:

http://arstechnica.com/media/news/2011/ ... mpaign=rss
Jeopardy: IBM's Watson almost sneaks wrong answer by Trebek
By Casey Johnston | Last updated a day ago

Watson, the IBM computer designed to take on humans in the quiz game Jeopardy, made its television debut last night. Positioned between two past Jeopardy champs, Ken Jennings and Brad Rutter, Watson's swirling globe avatar was able to hold its own, finishing the first round tied with Rutter at $5,000.

Dr. Chris Welty, a member of Watson's algorithms team, was on hand to provide commentary during Rensselaer Polytechnic Institute's showing of the Jeopardy episode at the school's Experimental Media and Performing Arts Center. Ars was there to hear his take.
No context

First, a quick rundown of how Watson plays Jeopardy. The computer is fed the answer in text form at the same time the answer panel appears to the two human players. Watson then queries its database for an appropriate question response, a process that doesn't involve using the Internet at all. Welty noted that game shows are federally regulated and there were two auditors present while the episode was filmed to make sure the computer wasn't querying the Internet for answers.

Watson then must push a physical buzzer to answer questions, just like its human competitors. While this would seem to be a task at which computers would have an overwhelming advantage, Welty noted that Rutter was so well-known for his lightning fast buzzing that the producers weren't even mildly concerned.

When the match began, the computer got off to a strong start: it took control of the board away from Rutter on the second turn, immediately nailed a Daily Double square, bet $1,000, and got the question right. But later, on a Name That Decade question, Jennings answered incorrectly with "what is the 1920s?" Watson, which can't see or hear and so can't pick up on the follies of its competitors, followed Jennings' answer with its own: "What is the 1920s?"

"No, Ken said that," Alex Trebek replied as the avatar's sphere turned orange with embarrassment.

During a commercial after Watson's decade gaffe, Welty noted that the team thought the ability to process other players' wrong answers would be unnecessary. "We just didn't think it would ever happen," Welty said, laughing.

Watson also tripped up on an "Olympic Oddities" question, but so imperceptibly that Alex Trebek didn't notice at first, raising an important point of clarification. After Jennings answered incorrectly that Olympian gymnast George Eyser was "missing a hand," Watson answered, "What is a leg?"

Welty said Trebek initially accepted Watson's answer, but the taping had to be stopped and the sequence reshot because Trebek had forgotten that Watson wasn't aware of the context created by Jenning's answer.

If a person had answered the Oddities question the way Watson did, they could have been presumed to be following the context of Jennings' answer, with the "missing"-ness of the leg implied. But since Watson couldn't have heard Jennings, its answer of "What is a leg?" rather than "What is missing a leg?" was actually deemed incorrect. In the aired version of the episode, Trebek declares Watson's answer wrong.

Last night's airing was the first of three and it covered only the first round of the game. Watson, Jennings, Rutter, and Trebek will continue tonight beginning with the double and final Jeopardy rounds of the first game, with a second full game to be played on the third night.
And the official website for it all, explaining how it all works http://www-943.ibm.com/innovation/us/watson/
JUSTICE, n A commodity which is a more or less adulterated condition the State sells to the citizen as a reward for his allegiance, taxes and personal service.
Sojourn
Registered User
Posts: 5649
Joined: 02 Sep 2004, 02:00
Location: Still looking...

Re: IBM Takes on Jeopardy.

Post by Sojourn »

I can understand chess vs a comp, the analytics gives it a somewhat fairer platform. (Although that was true in Big Blue's time... since the game was solved it is probably not true anymore.)

Proof it was solved in spoiler.
Spoiler (show)
"I was playing in a tournament in Germany one year when a man approached me. Thinking he just wanted an autograph, I reached for my pen, when the man made a startling announcement. 'I've solved chess!' I sensibly started to back away, in case the man was dangerous as well as insane, but the man continued: 'I'll bet you 50 marks that if you come back to my hotel room I can prove it to you.' Well, 50 marks was 50 marks, so I humored the fellow and accompanied him to his room."

"Back at the room, we sat down at his chess board. 'I've worked it all out, white mates in 12 no matter what.' I played black with perhaps a bit incautiously, but I found to my horror that white's pieces coordinated very strangely, and that I was going to be mated on the 12th move!"

"I tried again, and I played a completely different opening that couldn't possibly result in such a position, but after a series of very queer-looking moves, once again I found my king surrounded, with mate to fall on the 12th move. I asked the man to wait while I ran downstairs and fetched Emmanuel Lasker, who was world champion before me. He was extremely skeptical, but agreed to at least come and play. Along the way we snagged Alekhine, who was then world champion, and the three of us ran back up to the room."

"Lasker took no chances, but played as cautiously as could be, yet after a bizarre, pointless-looking series of maneuvers, found himself hemmed in a mating net from which there was no escape. Alekhine tried his hand, too, but all to no avail."

"It was awful! Here we were, the finest players in the world, men who had devoted our very lives to the game, and it was all over! The tournaments, the matches, everything - chess had been solved, white wins."

About this time Capa's friends would break in, saying "Wait a minute, I never heard anything about all this! What happened?"

"Why, we killed him, of course."
But random trivia questions? Hardly fair. Lets get into our souped-up GTI and race that Model-T.
Last edited by Sojourn on 17 Feb 2011, 10:02, edited 1 time in total.
RuadRauFlessa
Registered User
Posts: 20576
Joined: 19 Sep 2003, 02:00
Location: Bloodbank

Re: IBM Takes on Jeopardy.

Post by RuadRauFlessa »

Sojourn, I suspect there is some very, very, very, very,very intricate AI behind the app on that monster of a machine. It obviously has to "understand" our language in order to gauge a free form question... know what it is about... what is asked... what is required as a response... and to then formulate the response.
:rock: :rock: :rock: :rock: :rock: :rock: :rock: :rock: :rock: :rock:
Spoiler (show)
Intel Core i7-2600k @ 3.4GHz
Corsair Vengence 2x4GB DDR3 2000MHz
Thermaltake Toughpower 850W
ASUS nVidia GTX560 1GB
CoolerMaster HAF 932
Sojourn
Registered User
Posts: 5649
Joined: 02 Sep 2004, 02:00
Location: Still looking...

Re: IBM Takes on Jeopardy.

Post by Sojourn »

great.
intricate AI to dumb the AI down to the level where humans can (just barely) compete.
makes perfect sense.
Post Reply