How IBM Plans To Win Jeopardy! 154
wjousts writes "Technology Review is reporting on IBM's plans to take on Trebek at his own game. The 'Watson' computer system uses natural-language processing techniques to break down questions into their structural components and then search its database for relevant answers. A televised matchup with Trebek is planned for next year. 'David Ferrucci, the IBM computer scientist leading the effort, explains that the system breaks a question into pieces, searches its own databases for "related knowledge," and then finally makes connections to assemble a result. Watson is not designed to search the Web, and IBM's end goal is a system that it can sell to its corporate customers who need to make large quantities of information more accessible.'"
Dealing with Layered Problems (Score:5, Funny)
As a fairly avid though novice crossword puzzler, my mind explodes with questions. Could Watson discern a four letter word for "Pleasant French city" (Nice)? Or what about a four letter word for "Beefy Laker" (Kobe)?
Lastly, will Watson have something inane and boring to talk about during the break?
Alex Trebek: Now, Watson, it says here that you are named after Thomas J. Watson who forbade his employees to drink and even frowned upon it while off the job?
Watson: That is correct. It is against IBM regulation 4-245 Section 8 to consume alcohol on the premises of any facility.
Alex Trebek: Fascinating, I'm sure you've never broken that strict regulation, ha ha.
Watson: Good sir, I am a computer, drinking is not within my capacity.
Alex Trebek: Um, right. So could you tell us something interesting about yourself?
Watson: *pauses to search records* During the fabrication of my circuitry, several engineers went months without sleep. Leading one to go insane and killed his wife and kid before taking his own life in a double homicide/suicide case.
Alex Trebek: How unfortunate. Well, I wish you the best of luck today in Jeopardy.
Watson: Thank you, my snide game show master.
Jesus (Score:2, Funny)
What was an extra-terrestrial?
Jesus (Score:5, Funny)
What was an extra-terrestrial?
How tastelessly incorrect. Extra-terrestrials don't come back to life. Watson would cross reference The Bible with many recent movies and come up with the correct question we were looking for: "What was a zombie?"
Re: (Score:2)
"My kingdom is not of this world" indeed...
Re: (Score:3)
I know you were going for funny, but in reality Jeopardy never uses one(or two)-word answers. There are too many possible questions for a one-word answer.
Example:
A: Caesar
Q: Who crossed the Rubicon and said 'the die is cast'?
Q: Who said 'Et tu Brute?' when he was assassinated?
Q: Which Roman emperor adopted his great nephew Octavian who later became Augustus Caesar?
Q: What salad is made with Romaine lettuce, anchovies, garlic and lemon?
etc. etc.
Instead you use the long answer and get a short question:
A: This
Re: (Score:2)
They often do, but they're typically framed within a narrow, specific category. For example, the category might me "National Drinks" or some such thing. Typical Q/As might be Japan (Sake), Russia (Vodka), and so forth. Jeapoardy! also has a few other categories (anagrams, for example) that frequently use one or two words.
Re: (Score:2)
Re: (Score:3, Interesting)
Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.
An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don
Re:Dealing with Layered Problems (Score:5, Interesting)
Presumably they will either have to take into account the clues that come from the category itself (as in your example) or rig the system by avoiding "trick" categories. It's not an easy problem and it'll be very interesting to see what IBM come up with.
An example from last night, they had a category "Knockouts" in both the first and second round. In the first round, all the answers were hot women (i.e. knockouts!), in the second round all the answers were about boxing. How will Watson deal with this? I don't know.
Yes, there are categories which require the contestant to have an active imagination and it's these categories I wish the article had addressed instead of a vanilla one. And I believe it's these categories that makes Jeopardy fresh and new after decades.
... oh, say Pig Latin!
... as they could make this into an annual competition drawing fans and viewers much like the quest to beat the world chess grand masters.
In retrospect, I should have broke out the conversation into a different post so that this wasn't modded +5 Funny. I'm seriously interested in how IBM plans to address things that require the natural speech recognition of Alex Trebek. Does it take into account other answers in the same category to "catch on" like some contestants obviously do?
Then there's the folks running Jeopardy who could pick some categories that would wreck Watson and give the humans the creative advantage. I hope they exploit this creative ability humans have and write an entire category in
In reality, they stand to have much more to gain if the machine comes close to winning
Re:Dealing with Layered Problems (Score:4, Insightful)
Re:Dealing with Layered Problems (Score:5, Insightful)
It's possible that the questions for that particular show will be specifically chosen to be more explicit and less ambiguous ...
Yes, clues like "It's the cube root of 474552" would level the playing field.
Isn't the purpose of this to let Jeopardy be Jeopardy? And see if a computer can compete at what the show is?
Re:Dealing with Layered Problems (Score:5, Interesting)
Without speculating on the specifics of tweaking the AI, my guess is that IBM has tried to think through these things. Having put together a few AI bots myself (purely recreationally - you know, just for kicks), I know that I let them play in the real world for quite a while to work out the kinks before unveiling them to nerdy friends and family to show them off and demonstrate just how much time and sleep I'd wasted. My poker-bot played thousands of games in free online rooms before I told anyone that I was even working on him.
IBM has probably been feeding Watson DVR'd episodes for a while now so that they could identify (if not fix) the kind of gotchas that you're thinking about.
Re: (Score:2)
The most-discussed type of AI on that list is a "chatterbot" like ALICE or the classic ELIZA -- one that basically looks for key words and finds pre-written responses to them. That approach could probably tackle a variety of Jeopardy-type questions if someone took the trouble to feed the AI a bunch of sui
Re: (Score:1, Funny)
Re:Dealing with Layered Problems (Score:5, Informative)
Assume it's a perfect cube.
x^3 is 6 digits, so we're looking at numbers from about 50 to 100.
x^3 = 4XX
6^3 = 216
7^3 = 343
8^3 = 512
70 < x < 80
x^3 ends in an 2, so the cube root must end in an 8.
78.
Seriously though, square roots are easy peasy.
Cube roots let you use the awesome property that:
0 - 0
1 - 1
2 - 8
3 - 7
4 - 4
5 - 5
6 - 6
7 - 3
8 - 2
So you can always figure out the last digit of the cube root of a number VERY easily (no, you don't need to memorize that list).
Then you use the size of the number to get a range, and then estimate. If you're feeling ballsy, you can go for it. Spend the first few seconds (before people buzz in) and get your range down. Then buzz in and spend a couple seconds estimating, then answer (just say "what is..." right when you buzz in). If someone else buzzes in first, more time for you to think.
4th powers are just doing the square root twice.
The list for 5th power roots is neat, too.
0 - 0
1 - 1
2 - 2
3 - 3
4 - 4
5 - 5
6 - 6
7 - 7
8 - 8
9 - 9
0 - 0
Re: (Score:2)
Which trick?
If you're referring to finding out the last digit, it certainly DOESN'T always work out that nicely.
For squares we've got 3 and 7 both ending in 9. 8 and 2 both ending in 4. 9 and 1 both ending in 1. 4 and 6 both ending in 6.
Similar shit streams on down the ass for any even-power situation.
Does this trick hold true for all odd powers? I've got a feeling that it does, and would do so for any base. Couldn't be arsed to try to prove it though.
Re: (Score:2)
Doesn't work for 2 or 5...
2^41 = 2199023255552; % 100 = 52. 2 % 100 = 2.
52 != 2.
5^41 = 45474735088646411895751953125; % 100 = 25. 5 % 100 = 5.
25 != 5.
Re: (Score:3, Interesting)
Speech recognition?
The machine will be receiving a text file of the question.
Hell, I bet the thing is always the first to the buzzer too.
Re: (Score:3, Interesting)
It may not be... would it hit the buzzer and hope it had time to compute the answer like many people on those shows, or would it wait until it had time to compute, and then ring in only if it has the answer?
Re: (Score:2)
Seeing as it can read the question instantly, instead of having to wait for Trebeck to read it out (or read it as text at human speed), I think it's got a serious advantage.
I don't think the computational I/O speed is really going to be a limiting factor in this. The crux will be the dataset it has and whatever kind of strategy (being more or less gutsy with the buzzer, bidding more or less on the daily doubles / final round, etc. based on confidence of being right) it uses.
It's not like playing chess agai
Re: (Score:2)
it could buzz before it even has the answer calculated.
So can players. And they do quite frequently.
Re: (Score:2)
Well shit, I thought the 1st round was going to have knockouts (boxing) that occurred during the first round, and in the second round, the category would be about knockouts occurring in the second round.
Re: (Score:2)
Well knockout is a synonym for "hot babe". Shouldn't be too hard (of course I'm sure there will be something else that will be very hard for it)
CATS MEANING ALL RESPONSES START WITH CAT (Score:5, Insightful)
Alex will interject while reading the category such as "'Cats'--and that means all the words in this category start with 'Cat'."
Then the bot would read the closed caption that the category is "CATS MEANING ALL RESPONSES HAVE A WORD THAT STARTS WITH CAT" and include that in its reasoning. Then the clue "They are the popular makers of earth moving equipment" becomes something like "They are the popular makers of earth moving equipment, starting with 'CAT'".
Re: (Score:1)
Hm... one of the categories could be "Cats", you say...
That must be why IBM didn't want the machine to search the Internet!
- RG>
Re: (Score:2, Interesting)
Its an automated crossworld puzzle solver. How it works (and my advisor led the project, though I don't work on anything remotely similar) is that it has a large number of solver modules that are each good at a certain kind of clue. One might be really good at looking up famous people based on keywords. Another might be good at... I dunno some other type of crossword clue.
Then each
Re: (Score:2)
I wonder how they plan to do with . . . categories where words must be so many letters in length or perhaps start with certain things . . .
If they don't play one or more of these categories, it won't be full-up Jeopardy!(tm).
Re: (Score:2)
Seeing as you've thought of this by idly considering the problem, I would be extremely surprised if people who have spent months of research on the problem didn't already cover this one.
A test with Wolfram|Alpha (Score:2, Interesting)
I fed all the Jeopardy questions into Wolfram|Alpha and it got every single one right.
Only if... (Score:5, Funny)
It can answer in Sean Connery's voice and make your mother jokes at him.
Otherwise I'll probably pass and look up old SNL skits on youtube instead.
Re:Only if... (Score:5, Funny)
So I think IBM's plans here are to
Use a high-tech set of
Computers to create a
Knowledge processor that can be monetized.
I think
That wanting
To use such a
Rediculously advanced
Engineering marvel to make Sean Connery jokes would
Be a waste of
Everone's time, energy, and
Karma
Re: (Score:1)
think of it this way, if it can make jokes as well as being that good, I believe it would make more money and add some unexpected comic relief to things making it that much more valuable.
you don't need to put your A people on it, just some guys that know what they are doing may do it on their time as well...
Re: (Score:2)
They can't also have some fun with the snobbiest host along the way.
I never thought Trebek was snobby at all. Did I miss something?
Maybe Jeff Foxworthy is more your speed?
Re: (Score:3, Funny)
How IBM Plans To Win Jeopardy! (Score:1, Funny)
They plan to answer "Kebert Xela" and send that bastard back to the dimension where he belongs.
Suck it Trebek! (Score:5, Funny)
I wonder how well it'll do at Anal bum cover.
Re:Suck it Trebek! (Score:5, Funny)
Clearly, some mod has never seen SNL.
(That's 'An Album Cover', Connery!)
Re: (Score:2, Funny)
or the other classics...
Connery - "I'll take the PEN IS mightier for 300..."
Trebek - "That's the Pen Is Mightier..."
Connery - "I'll take Famous Tities for 500"
Trebek - "That's Famous Titles..."
It's not so easy when you don't have the answers to look at is it Trebek?
Re: (Score:1, Informative)
A: The answer is, what do you call someone who can spell neither penis nor titties.
Q: What is weszz?
Re:Suck it Trebek! (Score:3, Informative)
Thank Goodness for Hulu [hulu.com] (instead of finding dead Youtube Links)
The wiki entry is also pretty good: http://en.wikipedia.org/wiki/Celebrity_Jeopardy!_(Saturday_Night_Live) [wikipedia.org]
"Catch These Men" for "Catch the Semen".
"Things Trebek Sucks" over the actual category, "Potpourri".
Re: (Score:1)
I wonder how well it'll do at Anal bum cover.
I think "Anal bum jacket" is way funnier...
I get it now... (Score:5, Funny)
sell to its corporate customers who need to make large quantities of information more accessible.'"
They want to replace the call centres in India with call computers.
"Hello you're speaking to Susan Blue Gene how can I help you?"
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Well, those reps usually aren't empowered to do anything that the computer won't let them, so as long as the recognition is somewhat consistent, who cares?
Re: (Score:3, Funny)
Re: (Score:2)
The next one is even better: http://www.penny-arcade.com/comic/2000/1/31/ [penny-arcade.com]
Re: (Score:3, Funny)
Ah, but how are they to be mimicking the bizarre structure of grammar I am having to become accustomed to?
Please answer in the form of a question... (Score:2)
Making that statement in the form a question was appropriate there... unfortunately all the statements will be in the form of questions, with no answers in sight...
Is it plugged in?
Is it turned on?
Did you reboot?
Do you have your serial number?
Would you like me to drop this call under the guise of transferring you to someone who has no script to follow?
Re: (Score:2)
But if all those computers start malfunctioning, who can we call for tech support?
Wordplay (Score:5, Insightful)
A lot of Jeopardy questions are wordplay-dependent, something AI doesn't have the hang of yet (unless IBM has been toiling in secret on something truly amazing). Categories like "Rhyme Time" and questions like "Qhat does a Pharoah need when he has a cold?" (Answer: an Egyptian Prescription) are beyond the ken of a data search.
Many Jeopardy "answers" have the key to the answer within the question, though in some cases it may be enough to throw the program off. IE in a category like "Musicals" an answer like "Unlike his other hits, this musical wasn't 'the cat's meow' on Broadway." Raw data crunching will pair musicals, Broadway and "cats" but won't know where to go with "unlike." Only an aficionado will know that Andrew Lloyd Weber's "Starlight Express" tanked on Broadway.
So the writers, given any knowledge of the limitations of AI, can set a challenge which will be nearly impossible for current AI to meet. John Henry will live another day.
Re: (Score:2)
Right. That's why it's interesting. It needs to interpret the questions, construct a query, process and rank the results, as well as store and index all the information it needs for the game (no live connection to the internet).
If the questions aren't from the same group as usual, it won't be worth much. I would hope they wouldn't be specifically designed to either help or stump the computer.
Re: (Score:2)
Re: (Score:3, Informative)
Sorry, I used information from outside the original article without a citation. This is from the team's web info [ibm.com]:
... just like human competitors, Watson will not be connected to the Internet or have any other outside assistance.
Waste (Score:2, Interesting)
IBM is laying off American citizens, but hiring in Asia, and yet are spending all this money on gimmicks. This is the kind of thing that gives big companies bad names. Hopefully, as a consolation prize, the laid-off Americans can watch their former company go down in smoke on the game show, hoping it starts smoking and sparking like a cheesy Trek android meltdown.
Re: (Score:3, Interesting)
Re:Waste (Score:5, Informative)
Re: (Score:2)
I'm sorry, but you didn't answer in the form of a question.
Video Daily Double (Score:5, Interesting)
Alex: "Here to present the Video Daily Double is Harry Mudd, who always lies."
Harry: "I am lying."
Re: (Score:2)
Well, since a big portion of IBM is consulting (50%?), it probably makes sense for them to hire consultants who are citizens of the countries they will be working in... How many Americans are fluent in a second language? How many are willing to relocate for 5+ years to Asia? I'd guess that hiring local gives IBM a GOOD name in those markets.
Plus, the people they are hiring are human beings too. Why is hiring an American a more noble thing than hiring an Indian or Chinese person? They need to feed their
Why is "Watson" such a popular choice of name? (Score:1)
This isn't the first expert system I've come across called Watson and probably won't be the last.
But has anyone pointed out to these guys that Holmes was the smart one? Watson just tagged along with him like a faithful puppy and generally gave little help in solving the crimes.
So come on guys, how about a Holmes or Sherlock v1.0?
Re:Why is "Watson" such a popular choice of name? (Score:5, Informative)
Because Thomas J. Watson was the man who turned IBM into a global empire, and Thomas J. Watson Jr. brought it into computers. They successively held the top position at IBM for 57 years. So it's a very important name at IBM, and the connection with Sherlock Holmes is serendipitous.
Re: (Score:2)
I'd presumed that Watson refered to the assistant of Bell who first understood electric speech. Probably a wink to the quirky and confounding associations Jeapordy delights in.
Re: (Score:1, Informative)
http://en.wikipedia.org/wiki/Thomas_J._Watson [wikipedia.org]
Re: (Score:2)
Bad editors! (Score:5, Funny)
The summary clearly should have been titled "How does IBM plan to win Jeopardy?"
Re: (Score:2)
They will clearly win Jeopardy by patenting all the words in popular culture and using something like DMCA against all and any opponents... thus enforcing their patent argot.
The end? (Score:2)
A computer that can play Jeopardy?
THE END IS NEAR!
Re: (Score:2)
No, but you should start worrying as soon as it learns how to play Tic-Tac-Toe with itself. From there on, it's just a quick step into Global Thermonuclear War!
-dZ.
The Computer will lose (Score:1)
"I'm sorry, Watson. Your answer must be in the form of a question."
Infinite Loop (Score:1)
I hope "how many roads a can a man walk down..." is not a question
You'll Rue Day Trebeck!!! (Score:1)
I want to see how it responds to this question... (Score:4, Funny)
Re: (Score:3, Insightful)
Well, we already know that, it's 42.
The real question, is what is the real question for which 42 is the answer? That one is the tough one.
I suggest we build a planet, who's sole purpose is to calculate that question...
Re: (Score:2)
Wow. Seriously?
Ok, first, it was a joke, don't have a cow man.
Second, I have no clue why it was modded insightful, but whatever.
Third, no, phrasing answers in the form of a question is easy, have you evern watched Jeopardy before? That's the easiest part of the whole thing, "What is/are X" and "Who is/are X" are all you have to do the answers to phrase them as a question. No, the interesting thing will be if IBM's machine can parse the question in the correct context and come up with the correct answer fa
Re:I want to see how it responds to this question. (Score:2)
Answer: The number of minutes it would take for gravity-powered travel between antipodes, and the angle in degrees which causes a rainbow to appear.
Bullshit (Score:2)
The system is not designed to access the web?
Horse shit.
That huge fucking pile of data is getting in there from the web. It won't be accessing the web during the game, but it's still a fucking cache of random shit (mostly geography and world history) from the internet.
Re: (Score:2)
So? Your point? It's not accessing the web live, is it? No? So it's not getting it from the web.
The web isn't some magical information generating device. That information did not originate on the web, somebody put it there either from their own mind or from an offline location. By your own reasoning accessing the web isn't even "getting it from the web", as the web is just a huge cache of information from people's homes, schools, and even just plain their own minds.
Which is actually, in a sense, correct,
Re: (Score:2)
Uh, no.
A few interns were told to trawl wikipedia to grab data and form it into statements (probably prolog style).
I guaranfuckingtee it.
The game IS being rigged for the machine - the answers will be given to it in text. It won't have to press a physical button on a buzzer. Etc.
Re: (Score:2)
Ok, first, you totally missed my point. My point was that by extrapolating your objections to having a database of information obtained via the web, you actually get the data indirectly from the source, in some form. I don't see your objections to the human players who have been crawling for data for their own databases for the last 40+ years. Some of them have been crawling the web for data for the last 20 years!
It's the same damn thing, data is data, half of Jeopardy is knowing shit. That's an area co
Re: (Score:2)
No, you missed my point.
And you said so, yourself:
"So? Your point?"
Don't respond to my post, then put your own point in when you don't even get my point, then claim I don't get yours. Seriously, wtf?
Patents (Score:1)
They're doing it wrong (Score:1)
All they need to do is use their super computers to generate some digital footage of Alex Trebek engaged in beastiality (tappin' Rosie O'Donnell) and then tell him that if they win, the footage disappears forever.
It'd be far cheaper than what they are planning to do...and they can always leak the footage to youtube after they walk off with the winnings...
How IBM Plans To Win Jeopardy..... (Score:1)
By searching for all the answer on http://www.wikipedia.org/ [wikipedia.org] because we know all the information on that site is correct!
(Yes.. that was a joke)
Re: (Score:2)
Hey, anybody in the entire world can edit a wikipedia page, so you know you have the absolute best information possible!
Wrong bot for the job (Score:1)
Comment removed (Score:4, Funny)
Re: (Score:2)
It was more exciting (Score:2)
The first time I read it [slashdot.org]
Weird example (Score:2)
They claim they won't use Web data, but there's no way they can compile enough databases on their own to handle Jeopardy's general knowledge. Awards, lyrics, plots, characters... the list goes on and on and on.
WolframAlpha is a recent disappointment that's spent y
How to build a Jeopardy player (the easy way) (Score:2)
- Type entire "answer" as given on the board directly into google without quotes.
- Search the returned page for the most common word (ignoring 2 letter ones) in the titles of the pages.
- If the most common word appears more than 3 times, print "What is X?" where X is the common word.
- If no one term appears that often, don't ring in.
Voila. Instant human-crushing Jeopardy player.
Re: (Score:2)
You would think so, but it's not so simple.
Google results are far from useful without context information, especially with Jeopardy where the context is usually hidden by a pun or a play on words.
You just don't notice it because most people are able to process this contextual information on the fly, but it is a huge challenge for a machine to do it. There are litterally thousands of little bits of information that we collect as the answer is being given, including the context of the category, i.e. whether
Re: (Score:2)
Of course, you're solved it, you're so smart. Wait, I mean incredibly arrogant.
Re: (Score:2)
Easier...
Hide Ken Jennings inside.
IBM and PR (Score:1)
IBM has a history of inadvertently making terrible PR for themselves with these man-vs-machine stunts. Everyone here should remember Kasparov vs. Deep Blue. Expect IBM to win Jeopardy, and expect there to be a hailstorm of "IBM cheats" controversy after the game.
http://www.youtube.com/watch?v=cK0YOGJ58a0 [youtube.com]
Confidence (Score:2)
What I want to know is this:
The machine will probably be able to come up with an answer (maybe not the right one) much faster than all of the human opponents. But, what confidence will it have in that answer, and will it realize that a wrong answer will cost it?
Obviously if the machine just answers immediately (and no 'confidence' factor is involved) then it could provide wrong answers very quickly, and thus just lose money on every question as it "presses the button" to answer the question before the oppo
Alex is a chump. (Score:2)
Let's see if it can Win Ben Steins Money.
Re:Ben Stein is a chump. (Score:2)
Re: (Score:1)
That's S-words for four hundred...
Re: (Score:2)
Um, Trebek doesn't compete. He's the host of the show. The summary is stupid. The computer will be playing against other human contestants.