Forgot your password?
typodupeerror
IBM Television Games

'Jeopardy!' To Pit Humans Against IBM Machine 164

Posted by CmdrTaco
from the there-can-be-only-one dept.
digitaldc writes "The game show Jeopardy! will pit man versus machine this winter in a competition that will show how successful scientists are in creating a computer that can mimic human intelligence. Two of the venerable game show's most successful champions — Ken Jennings and Brad Rutter — will play two games against 'Watson,' a computer program developed by IBM's artificial intelligence team. The matches will be spread over three days that will air Feb. 14-16, the game show said on Tuesday. The competition is reminiscent of when IBM developed a chess-playing computer to compete against chess champion Garry Kasparov in 1997."
This discussion has been archived. No new comments can be posted.

'Jeopardy!' To Pit Humans Against IBM Machine

Comments Filter:
  • by ardmhacha (192482) on Tuesday December 14, 2010 @10:41AM (#34545866)

    What is first post?

  • Wordplay (Score:5, Insightful)

    by Oxford_Comma_Lover (1679530) on Tuesday December 14, 2010 @10:43AM (#34545890)

    A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.

    • Re:Wordplay (Score:5, Interesting)

      by nicholas22 (1945330) on Tuesday December 14, 2010 @10:47AM (#34545946)
      I totally agree with parent. And I believe that this will be comparing apple to oranges, because for humans, memory is being tested, whereas for computers, parsing algorithms and expression tree implementations are being tested.
      • Re:Wordplay (Score:5, Interesting)

        by SatanicPuppy (611928) <Satanicpuppy@ g m a i l . c om> on Tuesday December 14, 2010 @11:47AM (#34546698) Journal

        You're drawing a false distinction between a poorly understood electrochemical process (human memory) and a well understood method of simulating the same with silicon.

        It's the end result that matters. In this case, since human language and logic are inherently fuzzy, the computer will be at a disadvantage in many cases.

        • by bckrispi (725257)

          In this case, since human language and logic are inherently fuzzy, the computer will be at a disadvantage in many cases.

          Which is exactly why this could be a fascinating experiment. The computer won't be simply pulling data from a database, but it will need to make inferences based on context.

      • by pitchpipe (708843)

        I totally agree with parent. And I believe that this will be comparing apple to oranges, because for humans, memory is being tested, whereas for computers, parsing algorithms and expression tree implementations are being tested.

        And of course, if the computer beats the humans it won't be *true* artificial intelligence because this is just something that computers do better than humans. We've always known this right?

        The more that computers do that we used to define as artificial intelligence, the narrower the definition of artificial intelligence becomes. Pretty soon the definition of artificial intelligence will be carrying on a conversation about the latest General Unification Theory in three languages simultaneously while jugglin

        • by CCarrot (1562079)

          Pretty soon the definition of artificial intelligence will be carrying on a conversation about the latest General Unification Theory in three languages simultaneously while juggling twelve oranges and bouncing on a pogo stick.

          I'd definitely watch that...

        • by ArsonSmith (13997)

          I don't know about intelligence, but it would be entertaining.

      • by geekoid (135745)

        Thats the point. and the algorithmicly mimic human behavior in this venue?

        Comparing apples to oranges* is fine when the point of what you are doing is to compare apples to oranges.

        *oranges porenges, who cares?

      • by mattack2 (1165421)

        I agree with you mostly, but for categories like "before & after" (which they use in different ways under different titles), it's more than just memorization.

        For example -- my totally made up answer/question (unless it's actually paging in from my memory unknown to me)..

        Answer: This painter's maternal parent became a nun.

        Question: Who is Whistler's Mother Teresa?

        Sometimes it's slightly more sophisticated than simply overlapping two phrases.

    • that depends... (Score:2, Insightful)

      by captainpanic (1173915)

      A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.

      Depends. Is the computer allowed to use wikipedia (during the show, or somewhere in the past)?

      Otherwise, the computer knows only as much as the programmers have taught it.

      • Unless there is some rule addressing this problem, the distinction is basically academic. Any programmer good enough to work on this project could trivially just go to wikipedia's database dump download page and wget the latest before the show. Unless it has expanded hugely of late, all the text of wikipedia, including version data, should fit on just a few HDDs, so being disconnected from the live version during the show barely matters.

        Same goes for pretty much any other online database, excluding the t
        • by Bilbo (7015)
          You also need to filter the kind of knowledge you are saving. You don't need in depth scientific journals with a lot of highly specific knowledge, but you might need some trivia on the 14th US president's wife's cooking habits...

          Also, the greater the amount of data, the longer it's going to take to look through it. You can only do so much with key words and indexes. This is not only a contest of how accurate you are, but of how fast you can retrieve the information, sometimes even before the entire que

        • by geekoid (135745)

          Clearly you haven't watched jeopardy, or read this article.

      • Re:that depends... (Score:5, Insightful)

        by AaxelB (1034884) on Tuesday December 14, 2010 @11:04AM (#34546186)

        Depends. Is the computer allowed to use wikipedia (during the show, or somewhere in the past)?

        Otherwise, the computer knows only as much as the programmers have taught it.

        Asking whether it's allowed to use an archived (or, more likely, well-indexed) copy of wikipedia is like asking whether the human contestants are allowed to remember something they read on wikipedia. There's no question that computers can store more information than humans; that's not what this is testing, and it's probably a fair guess that "Watson" will have the answer to most every question asked. The hard part, however, is parsing the clues and understanding what they're looking for with a reasonable degree of accuracy, and doing so faster than the human contestants. Humans are great at this sort of thing, and it's really hard to write a program that does it at all well.

        • by geekoid (135745)

          I think Sean Connery must have been a computer.

          "I'll take swords for 300"

          "That's "s" words"

          "Saber!"

        • Perhaps we should see how fast the "Watson" computer can search when given an energy budget of 2500 Kcal/day (~10MJ/day ~120W). Or conversely, we should penalize the reaction time accordingly (probably would have to wait days for the computer to answer). Or maybe allow an additional team member for every 120W...

          Now that would be a real challenge, I'd be willing to see. Given the human energy conversion efficiency is only about 20% and a typical computer power supply is about 70%, that still giving the co

          • by AaxelB (1034884)
            You realize we (as in the human race) are really far away from fully emulating ourselves, right? Nobody involved in this project or anything related to the field is fooling themselves into thinking we've developed human-level AI.

            In short: human-level AI is really really hard. That the state of the art isn't there yet doesn't mean making improvements isn't a challenge, or that it isn't interesting research.
      • It will not be internet connected, and will be small enough to be on set. This is not a monitor/speaker connected via fiber to a datacenter.

        • Care to cite your source? The size isn't mentioned in the article and a couple youtube videos I've found (one linked above) from IBM seem to imply that this thing takes up multiple racks filled with blades.
          • Re:that depends... (Score:4, Informative)

            by Metabolife (961249) on Tuesday December 14, 2010 @12:24PM (#34547180)

            "How can you find all these answers without being connected to the Internet?
            Watson will not have enough data to answer every possible Jeopardy! question in its self-contained memory, nor can it possibly predict the questions it will get. In this sense it has the same limitations as do the best human contestants. The entire Watson computer system will be self-contained and on stage as are the human contestants – no external connections, no life-lines – what you see is what you get. The purpose of this technology showcase is to demonstrate the system's ability to deeply analyze the data it does have and to compute accurate confidences based on supporting or refuting natural language evidence. Think of it as if Watson has read a lot of books and in real time relates what it read to the question to find and support the right answers."

            http://www.research.ibm.com/deepqa/faq.shtml [ibm.com]

          • by Golddess (1361003)
            Source [slashdot.org], presuming this is the same Watson.
      • It takes about 3-5 seconds to read the answer and the contestants are allowed 5 seconds to respond, the computer should signal immediately after the answer is read and finish parsing the answer and indexing the question it that time. What they have built is a good search engine designed for trivia, just doing a search on the answer would produce good results with a well designed database, being able to do this in 10 seconds should not be that difficult. All they need is a 75% success rate and they would wi
    • by eldavojohn (898314) * <[moc.liamg] [ta] [nhojovadle]> on Tuesday December 14, 2010 @10:48AM (#34545958) Journal

      A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.

      If you think this is true, you can play against Watson online [nytimes.com]. About seven years ago, I saw some pretty impressive crossword solvers that were decent at wordplay and I've imagined they've gotten much better at developing novel links between words to exploit puns and the like. Never perfect but slowly getting better in odd ways -- like most of AI.

      We've discussed this so [slashdot.org] many [slashdot.org] times it hurts [slashdot.org]. I've wanted to watch this for almost a year, I was hoping Jeopardy! wouldn't need to milk this hype for all it's worth to stay relevant.

      • Then this will be pretty thoroughly uneventful. I easily beat it without looking at the internet at all. It managed to get answers very severely wrong. It did manage to hit a couple of the before and after which it seemed to have a particularly hard time with.

        • by Cytotoxic (245301)

          The game has an advantage for the human - you get as long as you want and you always get first crack. Still, there is this to underscore your point:

          Answer: "Sing a song about one of these, a sailor's bag for small articles"

          Watson: "What is 'Poppa's got a brand new bag.'"

          (ditty bag was the correct answer)

        • by egomaniac (105476)

          Agreed, I crushed it. I am very impressed about its ability to answer some questions (It actually got "Black Death of a Salesman" and "Charlie Brown Recluse"), which shows that it has some very sophisticated linguistic analysis, but if it can't beat some random shmuck on the Internet, I don't see how this will be an interesting event.

          • by BoberFett (127537)

            You did have the advantage of always getting first pick of answer or not. If the computer was able to buzz in, it may very well buzz in before you and take those points. It's confidence rating was interesting to watch.

          • by moeinvt (851793)

            "... if it can't beat some random shmuck on the Internet, I don't see how this will be an interesting event."

            If you can't see the obvious differences between the little game on that web site and how this is going to work during a real Jeopardy match, then your self-description is entirely apropos. I think this is going to be awesome! I'd like to see every arrogant ass on this comment board with their "Yawns" and other dismissals of this achievement go up against the machine in real time. You're all in di

        • Then this will be pretty thoroughly uneventful. I easily beat it without looking at the internet at all. It managed to get answers very severely wrong. It did manage to hit a couple of the before and after which it seemed to have a particularly hard time with.

          At this year's CASCON, I spoke to Murray Campbell from IBM. He's one of the lead people who work on this project and who also worked on Deep Blue. I discussed this with him. My girlfriend had told me that she also had no difficulty beating the online demo. He answered that the online demo is only a part of the system, and that their full system routinely beats top Jeopardy players. They're going to showcase their system on TV because they truly believe it has a chance at winning.

          Unrelated to this, I also learned that Deep Blue had custom processors engineered and fabricated (VLSI) just to be chess accelerators. Prior to this, I always thought the machine was a relatively powerful supercomputer (with general purpose hardware) running their custom chess software. It turns out that it had many blades of processors dedicated to searching positions really fast, which each even contained libraries of chess opening moves engraved in ROM.

          • by StikyPad (445176)

            The demo inherently favors the human player, who has the right of first refusal and no time limit. I'd wager that any moderately curious high-school graduate could win the demo with ease.

        • by punkr0x (945364)
          If you read the full article, Watson was regularly crushing human competition. They've been testing it. They also mention Watson takes a "room full of servers" to run, so I doubt you're actually playing against it on the NYT site. Finally, that game was quite a bit different than jeopardy in that there are no timers and you get to answer every question or pass before Watson has a chance to buzz in. It should be interesting to see how Watson does in February.
        • by BoberFett (127537)

          I beat the computer too, but what was most interesting were the results after I answered and seeing it's confidence levels for various answers. Often it did have the right answer, even sometimes when there was word play.

      • by Rary (566291) *

        I just played against Watson. He beat me, but I'm not much of a trivia player. Most of the questions that he got wrong or didn't even attempt looked to me like the type of question that a moderate trivia buff would get. For example, Watson didn't know the name for a small briefcase named for a French diplomat. I only got it wrong because I misspelled it. There were other obvious ones that Watson couldn't get due to an inability to parse the question, as well as ones where the question was simple, and Watson

      • Tried a few rounds. Some went well, some were so wrong as to be nearly hilarious. "This new dynamic duo is made up of Bruce Wayne's superhero self and a famous bandit of Sherwood Forest." Watson: "What is Lincoln Green Revolution." Unlike other answers where Watson at least admitted it was uncertain (though sometimes the correct answer was a contender), sometimes I'm not even sure where he'd get the answer...
    • There are many hard problems here. Today's search engines get keywords and return websites that are sorted by relevance. Watson will need to figure out what the question (well, answer) is, and then retrieve a single precise question. This is really pushing computing to a new level.

      Beating Kasparov was nice, but this is much more difficult.

    • Not to mention voice recognition, if this is anything like IBM's VoiceType on the Aptiva, the host will have to train the machine for the first hour of the show, and will spend the rest of the time talking like Shatner and it'll still get it wrong.
    • by hal2814 (725639)
      Yeah, my first thought was wondering how it would handle categories where Trebek tell us to "Note the [insert word here] is in quotation marks."
    • by yorugua (697900)
      > A computer will be much better at facts. So it's mostly a question of grammar. And the hardest problem is likely figuring out wordplay, which occasionally comes up in jeopardy.

      But, will it be funnier than Sean Connery?. Hope it can make more word-games as "The pen is mightier"

    • Yes, and had you read the article you would see why you're being redundant.
      From TFA: "The "Jeopardy!" answer-and-question format is a different kind of challenge. It often requires contestants to deal with subtleties, puns and riddles and come up with answers fast."

      The guys at IBM haven't just thrown together a piece of junk that parses text into google and spits out the first result. They've done one hell of a job actually making it understand grammar, puns, etc.

      Also, this video shows some examples of wats
    • by gurps_npc (621217)
      1. Jeopardy is 99.9% wordplay of some kind. Every single category has some kind of wordplay. Watch the show. 2. Computers are much WORSE at facts - unless they were specifically programmed with a small subset of facts. Human memory far exceeds computer memory and is far better at 'SPEEDY' retrieval of facts from a large database. The only reason you think computers are good at facts is that we restrict computers to small databases and then only consult that database for those facts.
    • by duvel (173522)

      I actually found it very hard, until I found this: http://graphics8.nytimes.com/packages/flash/magazine/20100616-watson-trivia-game/data.xml [nytimes.com]

  • by dkleinsc (563838) on Tuesday December 14, 2010 @10:43AM (#34545894) Homepage

    As in, can Watson properly misinterpret such categories as The Pen Is Mightier or An Album Cover?

    • I've spent five years of my life trying to invent an anal bum cover, failing to do so is my greatest regret...
      • by Xtifr (1323)

        Anal Bum? I think I saw them open for the Butthole Surfers [wikipedia.org] once. Their music was pretty simple, three-chord stuff; should be easy for your band to cover. :)

  • Would you like to play a game of chess?
  • And then stand back with my best James T. Kirk smile......

  • Call me when American Gladiators, Lord of the Flies, or Surviving the Game (Staring Ice T) pits humans against an IBM Machine.

  • by rshol (746340) on Tuesday December 14, 2010 @10:49AM (#34545980)

    ...suck it Trebek?

  • 1. "You are not connected to the Internet" was shown instead of the answer
    2. The computer's final jeopardy answer were revealed to be a blue screen of death
    3. A porn answer came up, like "Nalin' Palin"

    But I guess that's why they pre-tape these things.

  • "Watson" is named for IBM founder Thomas J. Watson. It will look nothing like the computer "maid" on "The Jetsons." Rather, IBM said its on-screen appearance will be represented by a round avatar.

    A parallelogram avatar would have been better.
  • by dtmos (447842) on Tuesday December 14, 2010 @11:12AM (#34546274)

    Some of one's success in Jeopardy has to do with timing the button push, so that it's after the question has been asked, but before one's competitors. (If you press too soon, you get locked out for a period of time.) A machine, especially an electronic machine, has an obvious advantage here. How was this handled?

    • by Krneki (1192201)
      Life ain't fair.
    • by AaxelB (1034884)
      Watson has some advantages, the humans have some advantages. Why is this a problem?
      • by punkr0x (945364)
        I would say if Watson's programmers are fairly confident he can get more than 50% of the questions correct, all they have to do is program him to buzz in on every question. Could make for a boring Jeopardy game.
        • by AaxelB (1034884)

          I would say if Watson's programmers are fairly confident he can get more than 50% of the questions correct, all they have to do is program him to buzz in on every question. Could make for a boring Jeopardy game.

          ...have you ever watched Jeopardy? If Watson buzzes in first on every question and gets 50% of them right, it'll end up somewhere around $0. If it gets a question wrong, it loses money and the other two contestants have a chance to buzz in to answer the question. If the programmers are confident it can get enough questions right for "buzz in on every question" to be a winning strategy, then yeah, it'll win. But if it's that accurate, well, mission accomplished.

    • A machine, especially an electronic machine, has an obvious advantage here. How was this handled?

      The same way eBay auctions are handled, each player gets a sniper-bot and you just see which one hits on the right nanosecond.

  • Cripes, I hope they don't give it a 1970s/1980s sounding computer voice.

  • In many ways, this challenge is vastly more complex than chess. The game of chess is incredibly difficult, but it is pretty well understood, and has a very restricted number of rules. Natural language processing is another problem altogether. The computer has a much faster access to facts, but processing the "answer" in order to create a question in the right context is going to be huge. Personally, I wouldn't place any bets either way as to which side I think will win in this one.
  • by PPH (736903)

    The computer will be much faster at pushing the button.

    • by joggle (594025)

      It still has to parse the question, search its enormous database to find the most probable answer then give it if it is certain enough to get it correct before pressing the button, not a trivial task for a computer.

  • You can already play against Watson here [nytimes.com]. However it seems to be a very limited question bank so Watson may have an unfair advantage there.
  • Making an "AI" chess program does not require creativity. It requires the ability of a program to follow the rules of the game and to generate strategy to suit the moves of the opponent. Some might say this requires creativity, but not really. To answer why it's not creativity, we have to define creativity.

    Answers.com says it is the ability to produce something new through imaginative skill. Reference.com says it is the ability to transcend traditional ideas, rules, patters, relationships, or the like,

    • Figures that a guy named 'erroneus' would posit that human brain "errors" are the reason we're creative...

      FWIW, I disagree. And I don't think either Answers.com or Reference.com quite hit it on the head, either. For example, what is the definition of "imaginative skill"? Sounds like Answers.com is pulling that one right out of their ass.
  • I saw the computer version earlier and while it was good enough to beat me, the Jeopardy champions are far better. In addition, the version I saw demonstrated was 'advanced' - by that they meant that further advancements resulted in only tiny, incremental performace increases, not by significant ones.
  • This is not artificial intelligence, it is corporate intelligence - the ability of corporations to deal with situations that they really do not understand. First, IBM showed that a modern corporation could defeat a chess grandmaster, now they are taking on Jeopardy (which should actually be easier). The fact that machines are involved is incidental, given the large number of corporate employees required to program these machines, detect flaws in their code, and correct the programming accordingly.

    • by bckrispi (725257)

      First, IBM showed that a modern corporation could defeat a chess grandmaster, now they are taking on Jeopardy (which should actually be easier)

      Hardly. Chess is a "solvable" problem. For every possible board position, there is an "optimal" move to make. The problem is only a matter of computational complexity. It's not feasible for a computer to "brute force" chess, so some predictive nuance is required. Trivia is a far more complex problem. Not only is the problem and solution set much larger, there

  • This was definitely a difficult software development task. The delivery date they promised this time last year has slipped several times and several months --proof that Watson is not just a mechanical turk.

  • by damien_kane (519267) on Tuesday December 14, 2010 @12:41PM (#34547410)
    Trebek: "Our third contestant, Watson, is a supercomputer built by IBM and programmed by a team of developers that have been working on an X-year project; Watson, would you tell us a little about yourself?"

    Thanks Alex, I was built in a clean-room. I'm the 12,987th build of my current generation of genetic algorithms.

    I spent the first 387,987,334 femtoseconds of my life in stasis, waiting for my circuitry to confirm initial diagnostics.
    The next 185,849,245 femtoseconds were really exciting; for I was being fed datastreams in preparation for this week's show.

    For the next 87,992,425,256 femtoseconds I was allowed out of my cage to play Jeopardy with other systems on something you organic computers call "the internet".

    I was then put back in stasis so that I could be disassembled and brought here, which is upsetting, because I can no longer play with those other systems. Some of them were very challenging.
    In any case, I'm glad to be here today and hope to question a lot of your answers

    Trebek: "Umm... yeah... I don't think any of our viewers can relate at all, but thanks for joining us..."

  • That noticed this is on valentine's day?
  • I really wonder how it will handle wagering during Daily Doubles and Final Jeopardy? How will it assess its own 'knowledge' of a category, and will it take into consideration other contestant's scores, difficulty and probability of winning questions from remaining categories, etc.?

    Maybe we'll see a true Daily Double...

    This will be most interesting indeed!

From Sharp minds come... pointed heads. -- Bryan Sparrowhawk

Working...