Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
AI Classic Games (Games) Google Games

Artificial Intelligence Bests Humans At Classic Arcade Games 148

sciencehabit writes The dream of an artificially intelligent computer that can study a problem and gain expertise all on its own is now reality. A system debuted today by a team of Google researchers is not clever enough to perform surgery or drive a car safely, but it did master several dozen classic arcade games, including Space Invaders and Breakout. In many cases, it surpassed the best human players without ever observing how they play.
This discussion has been archived. No new comments can be posted.

Artificial Intelligence Bests Humans At Classic Arcade Games

Comments Filter:
  • by penguinoid ( 724646 ) on Wednesday February 25, 2015 @05:25PM (#49132141) Homepage Journal

    Someone made a computer that's really good at reaction time, and at calculating trajectories.

    • Someone made a computer that's really good at reaction time

      It was done awhile ago. By IBM. Watch Watson play Jeopardy, and it is pretty obvious it won mainly because it was much faster at triggering the button. Watson wasn't better at answering the questions, it just got more chances.

      • At high levels, Jeopardy is all about who presses the button first. Watch a Tournament of Champions. All 3 people buzz in for pretty much every question.

        The trick is who can parse what the "answer" is really asking, recall the fact required, and then buzz in before the other players can do all 3 things. If Watson can do those things faster than a person, it won fair and square. Just being able to parse the "answer" was an incredibly impressive achievement.

        • by PRMan ( 959735 )
          It cheated. It was fed text instead of having to take the time (like the people did) to determine the meaning from the audible clue given.
          • The human players get the clue in text format also (printed on the monitor wall). Alex Trebek reading the clue aloud is strictly for the benefit of the mouth-breathers watching at home.

          • by Bonzoli ( 932939 )
            Yes, it has a great language/text codex, and the training process for watson is a scheduled batch job oddly enough. But once its trained its good at finding answers. I was disappointed they didn't have a good voice to text system running for the game.
            I've seen it in the lab at ibm and asked a lot of questions. Its a language codex which is quite good, sitting infront of a ranking database of information.
            The demo they had was medical journal based, and seemed quite useful for doctors that are lookin
        • ... and then buzz in before the other players can do all 3 things.

          This is not correct. It is NOT who buzzes first. Alex reads the question and then a light comes on. If you push the button before the light comes on, then you are locked out for a quarter of a second. So the trick is to push the button the instant the light flashes on. It is pure response time. Of course a computer is going to be better at that. That is 99% of the reason Watson won.

      • by gl4ss ( 559668 )

        a looong while ago.

        what do you think the military used the computers first for? I thought that was the joke of the comment.

        the thing is though giving it an arbitrary video game. making a robot to play just breakout is very easy, but that it plays breakout after it sees it is not that easy (though breakout is very easy on that scale, paddle follows ball and you get points, not much trial and error involved before you have winning combination(also wasn'tthat done a year or three ago already??)

    • Re:Breaking news! (Score:5, Insightful)

      by Dutch Gun ( 899105 ) on Wednesday February 25, 2015 @06:34PM (#49132743)

      Seriously, is there any doubt that a computer can easily defeat a human at a computer game that involves 95% pure reflexes and 5% strategy?

      The article shows a picture of Breakout, and tends to focus on the wrong things entirely... especially the title, trumping that "computers can beat humans". It's fairly impressive that computers can learn the rules of a simple videogame on their own and perform well, but beating humans is not exactly an apples to apples comparison, because while we can formulate strategies to maximize points, we're also prone to making simple mistakes due to our much poorer reflexes and coordination. So AI has a massive advantage with precision reflexes and calculations that it can make much faster than humans.

      Some of my previous jobs involved programmed AI game opponents for action games. As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead. I actually had do a lot of extra work to programmatically replicate the difficulties humans face when aiming at a moving target. However, collecting and processing global environmental knowledge and formulating complex strategies based on that knowledge is extremely difficult. That's why we typically build a lot of invisible hints into the environment itself for the benefit of AI, such as pathfinding-specific structures, or dynamic flags that signal potential rewards or danger. Even today, in many strategy games that involve complex ruleset (meaning brute force calculations can't work as well), the computer opponents inevitably have to cheat in order to compete with even modestly skilled players.

      Early videogames have very few of these sorts of challenges because of their largely static environments and the basic nature of the games. For the most part, you just need to formulate a few simple rules for an optimal victory condition, and when combined with a computer's incredible performance, you can easily trounce the best human players, simply because a computer never gets distracted, tired, or makes silly mistakes in judgement.

      Again, I'm not dissing the work the researchers did, which I found to be impressive, but the article and summary seem to be missing the point entirely by comparing them to human scores. It's fairly obvious that once a computer learns how to play with an optimal strategy, it's an absolute given that they'll score better than humans ever could.

      • by deesine ( 722173 )

        I figured out an endless pattern to Atari 2600 Space Invaders and PacMan, high score stuff. Was thrilled and disappointed to read about my solution in some Atari mag several years after my discovery.

        I figured I had beat the computer and was disappointed when War Games came out.

        --

        • I figured out an endless pattern to Atari 2600 Space Invaders and PacMan, high score stuff. Was thrilled and disappointed to read about my solution in some Atari mag several years after my discovery.

          I figured I had beat the computer and was disappointed when I wasn't asked to help defeat Xur and the Ko-Dan Armada.

          Fixed that for you.

      • by ljw1004 ( 764174 )

        The article shows a picture of Breakout, and tends to focus on the wrong things entirely... especially the title, trumping that "computers can beat humans". It's fairly impressive that computers can learn the rules of a simple videogame on their own and perform well, but beating humans is not exactly an apples to apples comparison, because while we can formulate strategies to maximize points, we're also prone to making simple mistakes due to our much poorer reflexes and coordination.

        Exactly. The article talks about the "advanced strategy" of tunneling a hole through to bounce the ball of the back wall. But that's only a useful strategy to make up for someone who doesn't have the reflexes to bounce the ball with their paddle, or can't be bothered. If the program had good reflexes and didn't get bored, then tunneling in breakout isn't any advantage.

        • That depends on the incentives the AI has.

          In this case, it appears it has incentives to gain the highest possible score as quickly as possible.

          In this case, tunneling and bouncing off the top wall better matches those goals.

          I read about his before and the computer starts out not knowing where the score is-- it has to learn which area is score and then do random things with the game until something succeeds at causing the score area to go up... and then optimize for high score and high speed.

          That sure sounds

          • by narcc ( 412956 )

            The bit about the score is key here. It's essentially no different than any other learning algorithm as it does not discover on its own that the goal is to achieve a high score. The computer vision part is neat, but nothing new, and ultimately does nothing to differentiate this from the zillion other similar projects as it is only used to find the score! Countless hobbyists and researchers have made ANN's and Genetic algorithms which produce similar results, both the computer vision part and the game-playin

      • by Lennie ( 16154 )

        I actually didn't see this story as news, I had seen a video of there work last year from before they were bought by Google.

        That same video was linked from the article:
        https://www.youtube.com/watch?... [youtube.com]

        What makes this more interresting is, they didn't tell the AI how to play the game, they let the AI learn to play the game on it's own.

        I think one of the things this what makes this also interresting is how few times the AI needed to learn the game and then also be good at it.

        • by PRMan ( 959735 )
          Pfft. It didn't take me 600 games of Breakout to learn to put the ball above the bricks...
      • Seriously, is there any doubt that a computer can easily defeat a human at ___________ that involves _____________?

        Of course not. Whenever a computer defeats a human easily, of course it isn't true AI. Computers were better at that all along. Leave that to computers so that humans can do the truly human work.

        • by narcc ( 412956 )

          Whenever a computer defeats a human easily, of course it isn't true AI.

          You're confused. I'm not sure how, exactly, but you might want to google "hard problem" and "strong AI" to net (ha!) yourself a better grounding.

      • by itzly ( 3699663 )

        As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead.

        It always makes me wonder why fights in futuristic movies are always done by people aiming by hand.

      • Some of my previous jobs involved programmed AI game opponents for action games. As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead.

        Then why did Steven Polge resort to making the ReaperBot cheat [mrelusive.com]?

    • by Anonymous Coward

      By telling the lie that you think this happened makes you a liar. Believing a liar makes you a liar. Are you smart enough to understand that? This did not happen. There is no proof. There is no video. The only evidence is a vague statement from a Republican. We all know their kind lies. Just look at Rmoney's claim that he is human. No. He is a man-like object. That is why he is so hateful. Again, you are a liar since you believe a liar.

    • by mrmeval ( 662166 )

      Also the game has finite limits which over time the Artificial Stupid can memorize completely enough to anticipate it ahead of time.

    • by rtb61 ( 674572 )

      Hate to break it to you but that was done, years and years ago, it is really easy for computers to do that stuff. The really hard bit it to analyse the environment and from that analysis create an internal virtual environment that you can base your calculations and optimum decisions on. It really is difficult for computers to analyse the visual environment, understand what is within that view and how the various elements will behave as changes occur. So virtual computer robotics is pretty easy because you

    • Re:Breaking news! (Score:4, Informative)

      by AmiMoJo ( 196126 ) * <mojo@w[ ]d3.net ['orl' in gap]> on Thursday February 26, 2015 @08:10AM (#49136015) Homepage Journal

      The key achievement here is that the AI was able to learn the game on its own in a relatively short time. Imagine if you had an industrial robot that could learn how to do tasks on its own and then modify its behaviour if the situation changed, and generally cope with a variety of situations.

      Also, they called it DQN which means "dumbass" in Japanese, so bonus points for that.

  • AI is now trivial pattern matching.

    • by AK Marc ( 707885 )
      For non trivial patterns, that is Intelligence. So they have trivial now, next they just need to improve that. A human does nothing but patten matching.
      • by Anonymous Coward

        Not true, we also do input and output. Gabage in, garbage out.

      • For example: I have pattern matched this thread^Warticle^Wweb site and decided it was a repetitive waste of time.

    • The classic arcade games are more of an issue of attention, than skill or thinking ahead.
      Once the AI figured out how the play the game, I bet it can focus and pay attention to the detail, more than a human can.

    • Don't even need that. I made pong for the TI-82 instead of paying attention in calculus class in high school, my "AI" could not be beat. Because it's really easy to do things when you can precisely calculate vectors and positions... It's actually harder to have something that makes human-like mistakes.

      I don't think actual breakout or space invaders would be significantly harder.

    • by s.petry ( 762400 )
      And Pay walled to boot!
  • I want my symbolic AI back...

  • Strategy games? (Score:3, Insightful)

    by gurps_npc ( 621217 ) on Wednesday February 25, 2015 @05:27PM (#49132169) Homepage
    No, merely reaction time.

    In other words, this is an example of good image recognition software, that's it.

    Show me a game that beat a human on a strategy based game, then you have something.

    • by fisted ( 2295862 )
      how about chess?
      • Humans play chess as a strategy game. Computers play chess as a math problem to solve by looking ahead x turns.

        "Go" on the other-hand must still be played as a strategy game by computers because looking ahead is not that helpful.

        • by Tablizer ( 95088 )

          Maybe humans memorize patterns of play (responses) that win then. If the computer cannot "look" far ahead in Go, than neither can people, most likely. What exactly is "strategy" that is different from predicting ahead and/or learning successful patterns?

    • Actually it does develop strategy. In BreakOut it doesn't merely bounce the ball with good reaction time. It learned to tunnel up to the back side of the wall. RTFA
    • by Trogre ( 513942 )

      To quote Queeg:
      Chess

    • by msobkow ( 48369 )

      It's not even a good example of image recognition, because the images to be processed don't have to be "understood" to be used. On top of that, the graphics of the games in question were very simple and primitive compared to what image recognition software deals with.

      Add to that the repetitive nature of old video games that were based on 99% reaction time and 1% strategy, and you can just flat out colour me "unimpressed" with this "research".

      Back in University, my AI project was a game player (a simpl

  • I like to see them play more advanced games such as Rainbow Islands, The New Zealand Story, Strider, or Zelda 3.
    • by Anonymous Coward

      Or say, Angry Birds or Fruit Ninja.

    • by Pieroxy ( 222434 )

      The real question being: did the AI enjoy it as much as a human?

      • by Twinbee ( 767046 )
        Not in my opinion because I don't think AI will ever experience qualia or anything for that matter.
    • by PRMan ( 959735 )
      Yeah, I've solved the first 3 of those. Have at it AI!
  • by Tablizer ( 95088 ) on Wednesday February 25, 2015 @05:32PM (#49132213) Journal

    Well, I outsourced my Donkey Kong playing before bots took it over, so there!

  • by Anonymous Coward

    This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.

    Training a neural net to play tetris is AI:101. Teaching it to play mario, has been done to death by AI students. Let me know when the AI complains about the ending of Mass Effect 3. Then I'll care.

    • > Let me know when the AI complains about the ending of Mass Effect 3.

      BUZZ CLICK WHIRRR... This Game Sucks. Click.

      (Side note - I wonder how long before the AI evolves (degenerates?) into comic book guy nerd speak... Are all of our nerdisms really just natural progressions of logic?
      BUZZ CLICK WHIRR... Worst Game EVER. Exclamation point. Exclamation point. Exclamation point. Click. )

    • Re:meh (Score:4, Funny)

      by Stormy Dragon ( 800799 ) on Wednesday February 25, 2015 @05:47PM (#49132359) Homepage

      Wasn't the AI complaining about the ending of Mass Effect 3 pretty much the plot of Mass Effect 3?

    • Re:meh (Score:5, Insightful)

      by vux984 ( 928602 ) on Wednesday February 25, 2015 @06:11PM (#49132549)

      This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.

      First the prime challenge in the games you are talking about is lining up a crosshair with a pixel with a mouse and selecting fire.

      If AI's had to do that they might have some difficulty. In practice the so-called AI bots already know where you are, and could keep their weapon lined up on your noggin through half the map without the need for line of sight. Tthey also get to target and fire at me without having to diddle around with a mouse or looking at the screen to see where I am.

      Get a bot to actually play such a game with the same UI and world view I have (keyboard and mouse and what they can see on screen and hear on the speakers) and they tend to be quite abysmal.

      Second, switch over to RTS games... and there the only way to give the AI any challenge is to stack the deck in its favor... whether its StarCraft or Supreme Commander or Wargame: Red Dragon. Or in a 4X game like Masters of Orion etc... we've yet to see an AI even really challenge a human being without giving it scripts to follow and extra resources to use.

      • by AK Marc ( 707885 )

        Second, switch over to RTS games... and there the only way to give the AI any challenge is to stack the deck in its favor... whether its StarCraft or Supreme Commander or Wargame: Red Dragon. Or in a 4X game like Masters of Orion etc... we've yet to see an AI even really challenge a human being without giving it scripts to follow and extra resources to use.

        Lots of extra resources, and a broken fog of war. And the easy/medium/hard on some games changes the damage/health of units.

    • by mjwx ( 966435 )

      This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.

      Thats because 99% of AAA games are twitch games. No strategy involved, just reflexes. Games like COD, Halo, et al. really limit what the player can do and it really is the fastest mouse wins. So they have to limit the reaction time of the AI to what a human is capable of.

      Now if you look at strategy games like Civ IV, you had to give the AI unfair advantages to put it on equal footing with human players. But turn based strategy games have a lot of fuzzy logic. So really an AI mastering COD is not news, we

  • by K3rn3lPan1c ( 2950877 ) on Wednesday February 25, 2015 @05:58PM (#49132469)
    Artificial Intelligence is no match for natural stupidity.
  • I'd like to see AI take me on after a few quality bong hits while I rock the orbits and ramps of Black Knight 2000 with perfect captive ball shots or I slam-tilt a Star Trek:TNG table without losing my 50 cents. ...Or totally get 57 second of playtime and 36,000 points out of another 2 games and 6 lost balls after I hit nothing but the glass and scoreless bumpers while every shot goes down the side gutters or perfectly in the center of a triple flipper gauntlet totally getting screwed over by those damn mag

    • by narcc ( 412956 )

      Damn, dude, you must be deaf, dumb, and blind.

  • This article reminds of this guy [cmu.edu] who did something similar a while back.
  • Now I can buy an AI so it can play the computer game I bought so I'd have more spare time?

  • ..it's a computer figuring out how to beat a computer at a kinda simple game

    The real world is a bit harder

    Still..well done!

  • Looks like the linked site is a pay wall or something. Renders the article in low res, throws a lot of pop ups. It seems to be a bad mash up of javascript running flash and pdf. Malware purveyors dream.

    Wonder why the editors let such bad sites and auto playing videos to be posted.

  • It's actually really cool that this happened, so it's a shame that most of the reporting on it is sort of "correction bait". The fact that it does good at these games without watching human strategies is interesting, but computers have strategies that humans lack, due to their increased reaction time (random thing happened, I can respond by doing X -> the computer is several orders of magnitude superior at this for free) and increased calculation time (the trajectory is curve such and such -> your vi

  • It looks like it worked with Atari 2600 games, which are ports of classic arcade games. A nitpick, but about 30 seconds playing the 2600 version versus the arcade version will show you a ludicrous level of difference betwixt. I don't want to belittle the work, but calling 2600 games arcade games is like calling a motorcycle a semi truck. Words have meaning- in this case, "Atari 2600 games" or "classic games". NOT arcade games.

  • Wow! For a generation, we knew that AIs are good at playing Global Nuclear War and now they tell us it's more like Super Mario Bros?

  • which game is the AI's favorite?

  • Unfortunately, the experiment came to an abrupt end when they threw "ET: The Extra Terrestrial" at the AI, whereupon after an hour of trying different tactics the AI decided that the only way to win was to send a power surge through the system, frying the only working Atari 2600 the researchers could dig up.

    This still classifies the AI as coming up with the best solution to the game ever implemented.

    Yaz

  • You gotta love humans. Every time an AI starts to be able to do something that was previously only our domain, it's all "Meh, wake me when..." and "Yeah, but a computer still can't..."

    Funny stuff.:)

    • by cfalcon ( 779563 )

      Programming a computer to ace these games was possible in the 90s, maybe the 80s. The reason this is interesting is HOW it taught itself, and how many of the games it could get good at (many of the games it could not learn). Cheerleading AI research is nice, but this isn't an example of a computer entering a new domain, this is a research example of something that can solve other problems in the field- an engineering demo of sorts.

      • Obviously. The learning is the "something that was previously only our domain," not the playing, which is precisely what people are reacting so defensively to, and what I find funny.

        Once AIs using ANN or whatever the ultimate technologies end up being can actually learn at a human level, it'll be "meh, wake me when it can appreciate a sunrise" or "Yeah, but a computer still can't fall in love!" My point is just that we move the bar in order to preserve our collective sense of being special snowflakes.

  • According to the paper, computer remains below human average level for Pacman. Does that means AI failed to reverse engineer Pacman's ghost behavior [gameinternals.com], including their bugs?
    • by cfalcon ( 779563 )

      That one is interesting because it took humans a ludicrous number of hours to figure out emergent patterns. It was trial and error (and the AI was deliberately not given an overly large amount of time on each game). Once the ninth key pattern was solved it became execution based. You could trivially code a machine to killscreen Pacman with no AI involved. Hell, you could probably do it with a very small Perl script- because humans already know the patterns that win.

      It WOULD have been interesting to giv

  • Well it's been 20 years and watson is still pressing his luck.

  • Screw Atari 2600 games. That's small fry gaming.

    Let this thing run full tilt boogie on a MetaTrader platform, and see what you get.

    • Well, AI has been on forex and stock market for a long while. And if you mean this specific implementation, it would probably be no match for the specialized solutions.
  • I got that the AI uses reinforcement learning, but how does it know whether it is doing well or badly? Even assuming ALL these games show score as big numbers on the screen: Did the AI come pre-equipped with a "layer" that parses the pixel data to read the score? Or did it learn to read numbers all on its own?? Because if it's the latter, that's pretty darn impressive, and I don't see any indication of the former on the article or the paper.
  • by Ihlosi ( 895663 ) on Thursday February 26, 2015 @02:59AM (#49135017)
    Reminds me of a competition to program a computer player for the classic Asteroids a few years ago.

    The best entries, however, didn't rely on AI, but on the fact that the RNG of the arcade game isn't random. Once the Asteroids-bot figured out the internal state of the RNG, it could basically use hyperspace to make targetted jumps (and never one that lead to the destruction of the ship), shoot at asteroids that haven't appeared yet and various other tricks. It was very impressive to watch one of these bots in action.

  • Of the 49 games tested there were about half which it did not do as well as a human player. They rated the performance of the AI against random play which equals 0 and a fairly skilled human player at 100%. The games the DQN agent did poorly at were:

    • Montezuma's revenge 0%
    • Private Eye 2%
    • Gravitar 5%
    • Frostbite 6%
    • Asteroids 7%
    • Ms. Pac-Man 13%
    • Bowling 14%
    • Double Dunk 17%
    • Seaquest 25%
    • Venture 32%
    • Alien 42%
    • Amidar 43%
    • Zaxxon 54%
    • ...

    It would be interesting to compare the games it did well at Vs those it did poorly at. Unfor

  • by itzly ( 3699663 )

    Now let's have an AI beat Zork.

    http://thcnet.net/zork/index.p... [thcnet.net]

  • The computer mostly figures out the game design jsut like human players, and couples that with super fast reaction time.

    The latter is the only reason it can beat the best human players.

    The bigger story is: what games did it suck at and why?

    • The games it sucked at are ones where progress is not tied to score. It uses a Q learner, so it relies upon having a numeric metric for success.
  • I will always beat the computer because I have more quarters :D
  • If not, then it is not intelligence, but just one program beating another program.
  • How they did this is a lot more interesting than the result. They're calling it a DQN, and the implications for automation tasks are awe inspiring.

    Here is a link to the source:

    https://sites.google.com/a/dee... [google.com]

Everyone can be taught to sculpt: Michelangelo would have had to be taught how not to. So it is with the great programmers.

Working...