Slashdot Log In
IBM Computer Program To Take On 'Jeopardy!'
Posted by
CmdrTaco
on Mon Apr 27, 2009 09:22 AM
from the i-wouldn't-do-that-alex dept.
from the i-wouldn't-do-that-alex dept.
longacre writes "I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human 'Jeopardy!' contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward. ... The team is aiming not at a true thinking machine but at a new class of software that can 'understand' human questions and respond to them correctly. Such a program would have enormous economic implications. ... The proposed contest is an effort by I.B.M. to prove that its researchers can make significant technical progress by picking "grand challenges" like its early chess foray. The new bid is based on three years of work by a team that has grown to 20 experts in fields like natural language processing, machine learning and information retrieval. ... Under the rules of the match that the company has negotiated with the 'Jeopardy!' producers, the computer will not have to emulate all human qualities. It will receive questions as electronic text. The human contestants will both see the text of each question and hear it spoken by the show's host, Alex Trebek. ... Mr. Friedman added that they were also thinking about whom the human contestants should be and were considering inviting Ken Jennings, the 'Jeopardy!' contestant who won 74 consecutive times and collected $2.52 million in 2004."
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Great... more phone-bots (Score:3, Funny)
Why employ real people when you can annoy the hell out of everyone who calls in by subjecting them to yet another tier of phone-bots.
Re:Great... more phone-bots (Score:4, Funny)
Why employ real people when you can annoy the hell out of everyone who calls in by subjecting them to yet another tier of phone-bots.
Or reverse it:
1) Use on telemarketers that call you. ...
2) Record
3)
4) Profit
Parent
The code name (Score:5, Funny)
Sources say the code-name for IBM's project is "Connery".
Trebek : This nobleman is believed to have written many of Shakespeare's works.
"Connery" : [pause] So that's your game, is it, Trebek? I was a coveted performer among the brothel ladies while you were still pissing your knee-pants, boy.
Trebek : Can one of the IBM people fix the computer?
"Connery" : Your mother's a whore. But don't feel badly, Trebek. She's not a very good one. Ha ha, ha ha!
Re:The code name (Score:5, Funny)
TREBEK: I'm sorry, that's "Japan US Relations." That's just awful and you know it.
Parent
Re:The code name (Score:5, Funny)
Alex Trebek: That's An album cover, not anal bum cover.
IBM: I can read, Trebek. That says Anal bum cover. I've spent five years of my life trying to invent an anal bum cover, failing to do so is my greatest regret
Parent
Re: (Score:3, Informative)
Alex Trebek: Yeah, it was a trick question, Mr. Connery. Why don't you pick a category?
Sean Connery: I've got to ask you about the Penis Mightier.
Alex Trebek: What? No. No, no, that is The Pen is Mightier.
Sean Connery: Gussy it up however you want, Trebek. What matters is does it work? Will it really mighty my penis, man?
Alex Trebek: It's not a product, Mr. Connery.
Sean Connery: Because I've ordered devices like that before - wasted a pretty penny, I don't mind telling you. And if The Penis Mightier works,
I'll take The Rapists for $300 (Score:4, Funny)
That's "Therapists"
Parent
Leap Forward? (Score:3, Insightful)
In what way would this be a leap forward? Looking up trivial facts in a database and spitting them out is easy, and not particularly significant...
Re:Leap Forward? (Score:5, Informative)
Parsing the questions in natural language, which is the goal here, is however very much *not* trivial. Doubly so since the clues and questions in a Jeopardy! game are usually at least somewhat obfuscated, contain puns, double entendres, etc...
Parent
Re:Leap Forward? (Score:5, Insightful)
Doubly so since the clues and questions in a Jeopardy! game are usually at least somewhat obfuscated, contain puns, double entendres, etc...
This is exactly why this sounds so implausible to me. You often have to take the category name and weave it in with the question (or rather, answer). A lot depends not on the knowledge, but on the phrasing of the "queries". Give me one example of translation software which can translate entire paragraphs well.
It makes me wonder how much "stress testing" they've done, by taking old Jeopardy questions and seeing if the output would be considered "correct" by a human arbiter.
Parent
Re: (Score:3, Funny)
Sure! From my handy-dandy English-to-Tech Manual-to-English translator:
According to precise how from unlikely sounding me hereto. Must needs question category name sewing needle rapprochement. Muc
Re: (Score:3, Insightful)
When Deep Blue went up against Kasperov, who could it practice against? Nobody.
That this got modded Insightful is the best argument yet for adding tags to /.
Re:Leap Forward? (Score:5, Informative)
That's the sort of thing that makes me believe that this team may be able to succeed.
When Deep Blue went up against Kasperov, who could it practice against? Nobody.
There are tens of thousands of Jeopardy! questions to go through before they start making up their own.
Well it did practice against other grandmasters, and it analyzed every game Kasparov had every played, where Kasparov went into the match blind.
Parent
Re: (Score:3, Insightful)
You don't need a human to determine who won a chess match. Winning is absolute.
You need a human to judge if the answer to a question, its phrasing, and its context were correct.
Re: (Score:3, Insightful)
Parsing the questions in natural language,
Natural language?
Outside of a Jeopardy! gameshow, I have never heard anybody use the this type of phrasing.
"This is a reason for you not handing in your homework Johnny"
"Why is because my dog ate it, sir"
Yeah, sounds very natural :-)
Re:Leap Forward? (Score:4, Insightful)
The leap forward is not in being able to look up facts in a database, it is in being able to interpret written questions properly.
There's a lot involved in interpreting natural language, and so far computers have been a far cry from being able to do it well. It says something that these algorithms are being tested against Jeopardy answers, since those are not completely natural language either -- they've been screened and edited to remove ambiguity.
Parent
Re:Leap Forward? (Score:5, Insightful)
I would think the challenge would lie in recognizing the question for what it is. E.g. "This playwright authored Hamlet" could confuse a computer - is it talking about Hamlet the play or literally a small town? Easy if you're human, not so easy if you're a computer. (From TFA: "The system must be able to deal with analogies, puns, double entendres and relationships like size and location, all at lightning speed.") I would imagine that the rhyming categories would be especially difficult.
Parent
Re: (Score:3, Interesting)
What msbmsb said. Also, this sounds similar to the problem already tackled (and aced) by Google Sets (or whatever they call it now). That's the feature where you give it some members of a set you have in mind (but you don't tell it what it's a set *of*) and it outputs more members from that set. For example, you give it "apple, orange, banana", and it gives you "grape, strawberry, lime". I'm guessing the way Google sets works is:
1) Run a search on all input phrases.
2) Find the most common statistically-
Re:Leap Forward? (Score:5, Funny)
Well, at least a computer program will bother to RTFA.
Parent
Re:Leap Forward? (Score:5, Insightful)
Being able to beat a human at Jeapordy is a fairly substantial subset of the Turing test sorted.
Natural language processing is an absolute and total bitch - take it someone who has studied it. One of my AI professors once explained it to me such; the human brain tricks you into believing the hardest tasks it accomplishes are the easiest. Stuff like language, walking, and so on take up far much more of your neural hardware than what you would consider 'thinking' - but it all happens subconsciously.
No, it isn't Artificial Intelligence per se - there is no real 'understanging' or 'intelligence' behind it -but it is a very serious technical challenge. There is a lot more to it than simply dumping Jeopardy questions into a standard search engines.
Don't take my word for it. Load up your favourite editor or IDE and start coding a simple chat bot. The difficulties that IBM must have overcome are best discovered through experience.
Parent
Re:Leap Forward? (Score:4, Funny)
Exactly what I was thinking.
Jinx! You owe me a Coke.
Parent
Re:Leap Forward? (Score:5, Insightful)
Tried it with "this playwright authored hamlet".
Didn't work:
http://shakespeareauthorship.com/imham.html [shakespear...orship.com]
If you just pulled words from the title, you might select "Oxfordian".
True, the answer is in the URL - but how do you know which to take?
Parent
Re: (Score:3, Insightful)
Re: (Score:3, Funny)
Re:Leap Forward? (Score:5, Informative)
According to TFA, the machine will get its questions as machine-readable text. The other human contestants will get it as text and audio. Also, the machine will not be connected to the internet.
Parent
Re:Leap Forward? (Score:4, Funny)
Sounds like they're guaranteed to win. The human contestants won't be given the questions at all :(
Parent
Re:Leap Forward? (Score:5, Insightful)
The step forward will be parsing the english language.
I hope it remembers to phrase its answers in the form of a question.
Parent
Re:Leap Forward? (Score:4, Informative)
I hope it remembers to phrase its answers in the form of a question.
Trivial. Word 1 of the question is "WHO" if answer is a person else "WHAT". Word 2 is "ARE" if answer is plural else "IS". Parsing Mr. Trebek's "answer" in three seconds, as you pointed out, is the hard part.
Parent
Wierd (Score:5, Funny)
Anyone else hearing "I Lost on Jeopardy" in their heads at the moment?
Re:Weird Al (or is it A.I.?) (Score:5, Funny)
Anyone else hearing "I Lost on Jeopardy" in their heads at the moment?
Now I am. Thanks for that. Jerk.
Parent
Hmm.. (Score:3, Funny)
I'd take Jennings (Score:4, Insightful)
The problem they might run into is the speed of pressing the button to respond. I would imagine the computer would be able to beat the human every time it knew the answer.
Re: (Score:3, Interesting)
The problem they might run into is the speed of pressing the button to respond. I would imagine the computer would be able to beat the human every time it knew the answer.
This is actually where I think the humans have an advantage. They can press the button just because they think that they WILL know the answer in the time allotted.
Watson may be designed to predict its own ability to answer. But to allow it to just press the button, then use the entire time limit to find it would not be fair...
Jeopardy doesn't work that way (Score:5, Funny)
I feel like someone should tell them how Jeopardy works... That thing isn't going to have too many questions to respond to.
Except at the "meet the contestant" part, maybe, which by the way should be fascinating.
I have a prediction for the meet the contestant... (Score:3, Funny)
Except at the "meet the contestant" part, maybe, which by the way should be fascinating.
"So, computer, you're about two months old, and you grew up in IBM's labs, right?"
"Bite my shiny metal ass"
This is how it starts (Score:5, Funny)
Re:This is how it starts (Score:5, Funny)
new objective: to win all human gameshows and use the prize money to buy off the entire planet instead.
I think Pinky and the Brain [wikipedia.org] already used that plot device.
Parent
Re: (Score:3, Funny)
Logs (Score:3, Interesting)
logs or it didn't happen! what they did to Kasparov was bullshit! Seriously if this magically gets better at 1/2 time, the least they can do is show the logs
Past Jeopardy questions (Score:3, Informative)
http://www.j-archive.com/listseasons.php [j-archive.com]
Anyone who thinks this is a trivial project has never watched Jeopardy.
While there are some of the typical "This is the capitol of Alaska" questions (answers), the real challenge (and the real money) is in the second round of the show where more ingenuity is usually required.
Re:Past Jeopardy questions (Score:5, Insightful)
Another good example of why Jeopardy would be difficult for an AI:
From Show #5680 - Friday, April 24, 2009:
http://www.j-archive.com/showgame.php?game_id=2989 [j-archive.com]
Category: SOUNDS LIKE A SNEEZE
Answer: To avoid or shun; a bumper sticker says to do it to "obfuscation"
There are a ton of synonyms for "avoid or shun". To pick the right one, you need to know that "eschew" sounds like a sneeze, which sounds like "ah-choo".
Anyone who thinks it is easy to write an AI to do that has probably never done it..
Parent
Buzzing In (Score:3, Interesting)
Can I have theatrics for $2000, Alex? (Score:3, Insightful)
So... why Jeopardy? IBM is trying to demonstrate software that can parse text for meaning. That's great. But there are plenty of other places/formats/etc. that you can demonstrate this technology. There are certainly far more useful applications of this sort of technology.
I'm guessing that they they are going after J! because...
1) The warm spotlight of a well-known TV show
2) There is still a lot of structure and form on J! that it's easier to achieve "success" than if they had the machine do something more free-form... e.g., read a novel and generate a plot summary or, heavens forbid, actually understand real human conversation
3) The computer could have other advantages, like impeccable buzzer timing (which is sometimes more important than actual knowledge, especially in the Tournament of Champions) and having memorized the material beforehand (the NYT indicates that it would have "read" study materials before the match), which also helps increase the likelihood of "success"
And to pile on the criticism of grandstanding, the machine will be fed electronic text. So no video camera to perform text recognition? No speech recognition (IBM afraid of the "wreck a nice beach" vs. "recognize speech" problem tripping up their theatrics?). And what use would this be? At least the AI text research done at Google is being put to good use, like improving their machine translation services. Aside from getting IBM's name plastered in the media, what exactly is this going to do?
game show is gonna be a real wopper (Score:3, Funny)
Mr. Trebek, would you like to play a game?
That's why IBM didn't buy Sun . . . (Score:4, Funny)
They had to make some tough investment decisions: Either buy Sun, or build a Jeopardy playing supercomputer, but not both.
I'm sure that the machine's performance in Final Jeopardy will awe us sufficiently, and IBM's management will be exonerated from walking away from Sun.
IBM: "Hey, Larry Ellison! Have fun in your toy sailboat! Call us when your database and hardware synergies can compete with us in pre-prime time light entertainment game shows!"
Rumor has it that HP is working on a massively parallel Intel supercomputer that can calculate the strategic advantage of bidding one dollar ($1) on "The Price is Right."
Head to head with IBM (Score:3, Funny)
We were about even going into final Jeopardy when he stubbornly refused to offer any question for the answer "Smartest ever computer in the movies". I got it right (HAL) and took the prize.
State of the Art Voice Recognition (Score:5, Interesting)
We were just talking about this in another thread... A lot o the comments here have been that natural language software isn't that great.
This isn't at all true. Today, understanding verbal and written communication is done by state of the art computers and programs at a rate about equal to human listeners and readers. Where a computer doesn't particularly excel is in parsing that language, mostly because a computer doesn't have access to our culture in order to absorb context, but context can still be added.
Here's an example from Ray Kurzweil's book "Age of Spiritual Machines." He talks about a phrase famously given to a language parsing program that goes thus: "Times flies like an arrow." This phrase can be understood in various ways:
"* The common simile: time moves quickly just like an arrow does;
* measure the speed of flies like you would measure that of an arrow (thus interpreted as an imperative) - i.e. (You should) time flies as you would (time) an arrow;
* measure the speed of flies like an arrow would - i.e. Time flies in the same way that an arrow would (time them);
* measure the speed of flies that are like arrows - i.e. Time those flies that are like arrows;
* all of a type of flying insect, "time-flies," collectively enjoys a single arrow (compare Fruit flies like a banana);
* each of a type of flying insect, "time-flies," individually enjoys a different arrow (similar comparison applies);
* A concrete object, for example the magazine, Time, travels through the air in an arrow-like manner."
(from: http://en.wikipedia.org/wiki/Natural_language_processing [wikipedia.org])
With a few facts it becomes obvious which is correct and which isn't. Tell the computer that there's no such thing as a 'time fly' and that flies don't time things, and that flies aren't sophisticated enough to like things in an affectionate manner, etc., and the correct interpretation soon becomes clear.
So, if you think a computer can't understand both written and verbal communication and then parse it quickly enough to answer the questions I will have to strenuously disagree. These challenges are quickly being overcome on the bleeding edge of the art. But since this perception persists that the state of the art is somehow bad, because Joe down the street messed with some free-ware language software that worked poorly -- I think a lot of people are in for a surprise, and winning Jeopardy in this manner is really the perfect way to show it off. Can't wait to see the Youtube clips.
Re:Is this what we really want? (Score:4, Funny)
Subjugated Human: What is my home?
Robot: That is the incorrect question. Please follow me to a "processing station".
Parent
Re: (Score:3, Insightful)
Actually, yes, it is. See, it's not just general knowledge, but, as about 9 billion other people in this thread pointed out, there are puns and other wordplay often involved.
What do you think the proper Jeopardy answer to this question is (in the category "Much Ado" for $100):
"It's the spirit that gets things done."
Answer: What is "can do"
The $500 version might be something like, "This recently hip-again party favorite was first created in New York."
Answer: What is "fondue"
Both of those are pretty easy exam
Re:Not AI just Google with a filter (Score:4, Insightful)
No argument will show you how wrong you are - please just try to code something similar. Or even a chat bot. Things like language processing and image recognition seem easy to humans because most of your brains activity is hidden from your conscious mind. Try making a computer do these things, and you will discover that what IBM is nontrivial.
Neither was selling calculating equipment to the Nazis either, like you said.
Parent