Artificial Intelligence Bests Humans At Classic Arcade Games 148
sciencehabit writes The dream of an artificially intelligent computer that can study a problem and gain expertise all on its own is now reality. A system debuted today by a team of Google researchers is not clever enough to perform surgery or drive a car safely, but it did master several dozen classic arcade games, including Space Invaders and Breakout. In many cases, it surpassed the best human players without ever observing how they play.
Breaking news! (Score:5, Funny)
Someone made a computer that's really good at reaction time, and at calculating trajectories.
Re: (Score:2)
Someone made a computer that's really good at reaction time
It was done awhile ago. By IBM. Watch Watson play Jeopardy, and it is pretty obvious it won mainly because it was much faster at triggering the button. Watson wasn't better at answering the questions, it just got more chances.
Re: (Score:3)
At high levels, Jeopardy is all about who presses the button first. Watch a Tournament of Champions. All 3 people buzz in for pretty much every question.
The trick is who can parse what the "answer" is really asking, recall the fact required, and then buzz in before the other players can do all 3 things. If Watson can do those things faster than a person, it won fair and square. Just being able to parse the "answer" was an incredibly impressive achievement.
Re: (Score:2)
Re: (Score:3)
The human players get the clue in text format also (printed on the monitor wall). Alex Trebek reading the clue aloud is strictly for the benefit of the mouth-breathers watching at home.
Re: (Score:2)
I've seen it in the lab at ibm and asked a lot of questions. Its a language codex which is quite good, sitting infront of a ranking database of information.
The demo they had was medical journal based, and seemed quite useful for doctors that are lookin
Re: (Score:3)
... and then buzz in before the other players can do all 3 things.
This is not correct. It is NOT who buzzes first. Alex reads the question and then a light comes on. If you push the button before the light comes on, then you are locked out for a quarter of a second. So the trick is to push the button the instant the light flashes on. It is pure response time. Of course a computer is going to be better at that. That is 99% of the reason Watson won.
Re: (Score:2)
a looong while ago.
what do you think the military used the computers first for? I thought that was the joke of the comment.
the thing is though giving it an arbitrary video game. making a robot to play just breakout is very easy, but that it plays breakout after it sees it is not that easy (though breakout is very easy on that scale, paddle follows ball and you get points, not much trial and error involved before you have winning combination(also wasn'tthat done a year or three ago already??)
Re:Breaking news! (Score:5, Insightful)
Seriously, is there any doubt that a computer can easily defeat a human at a computer game that involves 95% pure reflexes and 5% strategy?
The article shows a picture of Breakout, and tends to focus on the wrong things entirely... especially the title, trumping that "computers can beat humans". It's fairly impressive that computers can learn the rules of a simple videogame on their own and perform well, but beating humans is not exactly an apples to apples comparison, because while we can formulate strategies to maximize points, we're also prone to making simple mistakes due to our much poorer reflexes and coordination. So AI has a massive advantage with precision reflexes and calculations that it can make much faster than humans.
Some of my previous jobs involved programmed AI game opponents for action games. As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead. I actually had do a lot of extra work to programmatically replicate the difficulties humans face when aiming at a moving target. However, collecting and processing global environmental knowledge and formulating complex strategies based on that knowledge is extremely difficult. That's why we typically build a lot of invisible hints into the environment itself for the benefit of AI, such as pathfinding-specific structures, or dynamic flags that signal potential rewards or danger. Even today, in many strategy games that involve complex ruleset (meaning brute force calculations can't work as well), the computer opponents inevitably have to cheat in order to compete with even modestly skilled players.
Early videogames have very few of these sorts of challenges because of their largely static environments and the basic nature of the games. For the most part, you just need to formulate a few simple rules for an optimal victory condition, and when combined with a computer's incredible performance, you can easily trounce the best human players, simply because a computer never gets distracted, tired, or makes silly mistakes in judgement.
Again, I'm not dissing the work the researchers did, which I found to be impressive, but the article and summary seem to be missing the point entirely by comparing them to human scores. It's fairly obvious that once a computer learns how to play with an optimal strategy, it's an absolute given that they'll score better than humans ever could.
Re: (Score:2)
I figured out an endless pattern to Atari 2600 Space Invaders and PacMan, high score stuff. Was thrilled and disappointed to read about my solution in some Atari mag several years after my discovery.
I figured I had beat the computer and was disappointed when War Games came out.
--
Re: (Score:3)
I figured out an endless pattern to Atari 2600 Space Invaders and PacMan, high score stuff. Was thrilled and disappointed to read about my solution in some Atari mag several years after my discovery.
I figured I had beat the computer and was disappointed when I wasn't asked to help defeat Xur and the Ko-Dan Armada.
Fixed that for you.
Re: (Score:2)
The article shows a picture of Breakout, and tends to focus on the wrong things entirely... especially the title, trumping that "computers can beat humans". It's fairly impressive that computers can learn the rules of a simple videogame on their own and perform well, but beating humans is not exactly an apples to apples comparison, because while we can formulate strategies to maximize points, we're also prone to making simple mistakes due to our much poorer reflexes and coordination.
Exactly. The article talks about the "advanced strategy" of tunneling a hole through to bounce the ball of the back wall. But that's only a useful strategy to make up for someone who doesn't have the reflexes to bounce the ball with their paddle, or can't be bothered. If the program had good reflexes and didn't get bored, then tunneling in breakout isn't any advantage.
Re: (Score:2)
That depends on the incentives the AI has.
In this case, it appears it has incentives to gain the highest possible score as quickly as possible.
In this case, tunneling and bouncing off the top wall better matches those goals.
I read about his before and the computer starts out not knowing where the score is-- it has to learn which area is score and then do random things with the game until something succeeds at causing the score area to go up... and then optimize for high score and high speed.
That sure sounds
Re: (Score:3)
The bit about the score is key here. It's essentially no different than any other learning algorithm as it does not discover on its own that the goal is to achieve a high score. The computer vision part is neat, but nothing new, and ultimately does nothing to differentiate this from the zillion other similar projects as it is only used to find the score! Countless hobbyists and researchers have made ANN's and Genetic algorithms which produce similar results, both the computer vision part and the game-playin
Re: (Score:3)
I actually didn't see this story as news, I had seen a video of there work last year from before they were bought by Google.
That same video was linked from the article:
https://www.youtube.com/watch?... [youtube.com]
What makes this more interresting is, they didn't tell the AI how to play the game, they let the AI learn to play the game on it's own.
I think one of the things this what makes this also interresting is how few times the AI needed to learn the game and then also be good at it.
Re: (Score:2)
Re: (Score:2)
Seriously, is there any doubt that a computer can easily defeat a human at ___________ that involves _____________?
Of course not. Whenever a computer defeats a human easily, of course it isn't true AI. Computers were better at that all along. Leave that to computers so that humans can do the truly human work.
Re: (Score:2)
Whenever a computer defeats a human easily, of course it isn't true AI.
You're confused. I'm not sure how, exactly, but you might want to google "hard problem" and "strong AI" to net (ha!) yourself a better grounding.
Re: (Score:2)
As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead.
It always makes me wonder why fights in futuristic movies are always done by people aiming by hand.
Re: (Score:2)
Some of my previous jobs involved programmed AI game opponents for action games. As anyone who's faced an aim-bot knows, there's no real challenge for computers to perform many of the tasks humans find difficult, like putting a bullet through a moving target's forehead.
Then why did Steven Polge resort to making the ReaperBot cheat [mrelusive.com]?
Re: (Score:1)
By telling the lie that you think this happened makes you a liar. Believing a liar makes you a liar. Are you smart enough to understand that? This did not happen. There is no proof. There is no video. The only evidence is a vague statement from a Republican. We all know their kind lies. Just look at Rmoney's claim that he is human. No. He is a man-like object. That is why he is so hateful. Again, you are a liar since you believe a liar.
Re: (Score:2)
Also the game has finite limits which over time the Artificial Stupid can memorize completely enough to anticipate it ahead of time.
Re: (Score:2)
Hate to break it to you but that was done, years and years ago, it is really easy for computers to do that stuff. The really hard bit it to analyse the environment and from that analysis create an internal virtual environment that you can base your calculations and optimum decisions on. It really is difficult for computers to analyse the visual environment, understand what is within that view and how the various elements will behave as changes occur. So virtual computer robotics is pretty easy because you
Re:Breaking news! (Score:4, Informative)
The key achievement here is that the AI was able to learn the game on its own in a relatively short time. Imagine if you had an industrial robot that could learn how to do tasks on its own and then modify its behaviour if the situation changed, and generally cope with a variety of situations.
Also, they called it DQN which means "dumbass" in Japanese, so bonus points for that.
Lower the bar further. (Score:2)
AI is now trivial pattern matching.
Re: (Score:2)
Re: (Score:1)
Not true, we also do input and output. Gabage in, garbage out.
Re: (Score:2)
Re: (Score:2)
For example: I have pattern matched this thread^Warticle^Wweb site and decided it was a repetitive waste of time.
Re: (Score:2)
Re: (Score:3)
Re: (Score:2)
You've missed the joke.
Re: (Score:2)
The classic arcade games are more of an issue of attention, than skill or thinking ahead.
Once the AI figured out how the play the game, I bet it can focus and pay attention to the detail, more than a human can.
Re: (Score:3)
Don't even need that. I made pong for the TI-82 instead of paying attention in calculus class in high school, my "AI" could not be beat. Because it's really easy to do things when you can precisely calculate vectors and positions... It's actually harder to have something that makes human-like mistakes.
I don't think actual breakout or space invaders would be significantly harder.
Re: (Score:2)
Nooooooo (Score:2)
I want my symbolic AI back...
Strategy games? (Score:3, Insightful)
In other words, this is an example of good image recognition software, that's it.
Show me a game that beat a human on a strategy based game, then you have something.
Re: (Score:2)
Re: (Score:2)
"Go" on the other-hand must still be played as a strategy game by computers because looking ahead is not that helpful.
Re: (Score:1)
Maybe humans memorize patterns of play (responses) that win then. If the computer cannot "look" far ahead in Go, than neither can people, most likely. What exactly is "strategy" that is different from predicting ahead and/or learning successful patterns?
Re: (Score:2)
A pic^H^H^H link or it didn't happen.
Re: (Score:1)
Go atse?
Re: (Score:2)
Re: (Score:2)
To quote Queeg:
Chess
Re: (Score:3)
It's not even a good example of image recognition, because the images to be processed don't have to be "understood" to be used. On top of that, the graphics of the games in question were very simple and primitive compared to what image recognition software deals with.
Add to that the repetitive nature of old video games that were based on 99% reaction time and 1% strategy, and you can just flat out colour me "unimpressed" with this "research".
Back in University, my AI project was a game player (a simpl
Let it try at 80s/90s games (Score:2)
Re: (Score:1)
Or say, Angry Birds or Fruit Ninja.
Re: (Score:2)
The real question being: did the AI enjoy it as much as a human?
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
I'd like to see a robot play fruit ninja for real. Sure, it'd be a little scary having a robot waving a razor sharp katana around, in your kitchen, but think of the time you could save making fruit salad!
Re: (Score:2)
I want to see a computer play Mornington Crescent.
Re: (Score:2)
import random
import urllib.request
player = random.choice(['You', 'I'])
station = str()
tube_stations = urllib.request.urlopen(r'https://raw.githubusercontent.com/rk295/tube-postcodes/master/tube-postcodes.txt').readlines()
tube_stations = [i.decode('utf-8').split(',')[0] for i in tube_stations]
while True:
if 'You' == player:
station = input('Enter a station: ')
else:
I'm one step ahead (Score:4, Funny)
Well, I outsourced my Donkey Kong playing before bots took it over, so there!
meh (Score:1)
This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.
Training a neural net to play tetris is AI:101. Teaching it to play mario, has been done to death by AI students. Let me know when the AI complains about the ending of Mass Effect 3. Then I'll care.
Re: (Score:2)
> Let me know when the AI complains about the ending of Mass Effect 3.
BUZZ CLICK WHIRRR... This Game Sucks. Click.
(Side note - I wonder how long before the AI evolves (degenerates?) into comic book guy nerd speak... Are all of our nerdisms really just natural progressions of logic?
BUZZ CLICK WHIRR... Worst Game EVER. Exclamation point. Exclamation point. Exclamation point. Click. )
Re:meh (Score:4, Funny)
Wasn't the AI complaining about the ending of Mass Effect 3 pretty much the plot of Mass Effect 3?
Re:meh (Score:5, Insightful)
This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.
First the prime challenge in the games you are talking about is lining up a crosshair with a pixel with a mouse and selecting fire.
If AI's had to do that they might have some difficulty. In practice the so-called AI bots already know where you are, and could keep their weapon lined up on your noggin through half the map without the need for line of sight. Tthey also get to target and fire at me without having to diddle around with a mouse or looking at the screen to see where I am.
Get a bot to actually play such a game with the same UI and world view I have (keyboard and mouse and what they can see on screen and hear on the speakers) and they tend to be quite abysmal.
Second, switch over to RTS games... and there the only way to give the AI any challenge is to stack the deck in its favor... whether its StarCraft or Supreme Commander or Wargame: Red Dragon. Or in a 4X game like Masters of Orion etc... we've yet to see an AI even really challenge a human being without giving it scripts to follow and extra resources to use.
Re: (Score:2)
Second, switch over to RTS games... and there the only way to give the AI any challenge is to stack the deck in its favor... whether its StarCraft or Supreme Commander or Wargame: Red Dragon. Or in a 4X game like Masters of Orion etc... we've yet to see an AI even really challenge a human being without giving it scripts to follow and extra resources to use.
Lots of extra resources, and a broken fog of war. And the easy/medium/hard on some games changes the damage/health of units.
Re: (Score:2, Informative)
Seriously, you don't sound like you know what you're talking about.
Nor do you. Your AI doesn't have the same control interface as human players. An AI would never miss a stationary target unless you programmed it to or are really bad at programming. Humans do miss them. Humans have to deal with moving a mouse an exact distance. AIs don't. They simply move the exact distance required. No need to deal with mouse physics, no need to deal with running out of desk space, no need to estimate the amount of movement, etc... They don't need think which key does what then pr
Re:meh (Score:5, Interesting)
When Id released Quake C, this "you have to dumb down the AI" idea became very apparent. Someone wrote a replacement AI for the enemies that allowed them to learn and communicate. They had to follow the rules and physics of the game - so if they were within earshot they could communicate your position to each other, otherwise they couldn't.
The first couple of interactions would be pretty easy kills. Then one enemy would see that you were better armed and run away. That would be the last enemy you would see for a while. Then, when you were in a vulnerable position, the entire population of the level would ambush you in a coordinated attack. Game over. They were way, way, way too smart to be beaten. It was pretty fun to explore their learning capabilities and watch how they would win. But it didn't make for engaging gameplay, unless you are a complete masochist.
The same AI was applied to deathmatch player bots. They had no prior knowledge of the level, or strategies for playing the game. The first few kills were very easy as they figured out what to do. But as they learned your tendencies, they would very quickly evolve into a circle-strafe master. They also learned the map layout pretty quickly, including drop sites and periods for weapons and health. They would then time their circle-strafe to always be on the spawn site immediately as the health or ammo spawned. They would invariably win against even the best human players by monopolizing all of the supplies and winning a war of attrition. Very impressive to watch.
This "AI" program was very rudimentary, and it was already much too difficult for human players, despite being limited to the same "in game" knowledge and input capabilities as the human players. It makes perfect sense that the challenge and complexity of programming the AI for games mostly revolves around "dumbing down" the AI in a way that makes the enemies challenging and interesting, but also the right amount of "beatable".
Re: (Score:3)
The same AI was applied to deathmatch player bots. They had no prior knowledge of the level, or strategies for playing the game. The first few kills were very easy as they figured out what to do. But as they learned your tendencies, they would very quickly evolve into a circle-strafe master. They also learned the map layout pretty quickly, including drop sites and periods for weapons and health. They would then time their circle-strafe to always be on the spawn site immediately as the health or ammo spawned. They would invariably win against even the best human players by monopolizing all of the supplies and winning a war of attrition. Very impressive to watch.
Yes, and it gives another example of the issue. The AI has access to a perfect clock, and perfect spatial awareness, and has perfect control over its movement, sees everything in its field of view no matter how briefly it sees it...
It can run and jump a maze of catwalks over lava with perfect reliability - BACKWARDS. It can do all that and land on a specific set of coordinates where it knows supplies are going to respawn the millisecond it respawns, because it can time to the millisecond when it last picked
Re: (Score:3)
That echos my point, somewhat. It is pretty easy to design an AI for a lot of video games that can beat a human (without cheating). The AI code from Quake was only a couple of pages. Whether you use the moniker "constraints" or call it "dumbing down", it would take a lot more code to give the AI more human-like abilities. Probably several times as much code.
The same goes for the enemies. The code that gave them a degree of autonomy and communication was pretty small, but it made them unstoppable becaus
Re: (Score:3)
That echos my point, somewhat. It is pretty easy to design an AI for a lot of video games that can beat a human (without cheating).
Yes, albeit, for a slightly strained definition of "without cheating" :)
Whether you use the moniker "constraints" or call it "dumbing down",
I think the distinction is important. Dumbing it down would be deliberately sabotaging its ability to make good decisions. The constraints certainly have the same effect, but we aren't sabotaging the decision making itself, but merely restricting the information it actually has to work with access to human levels.
No perfect clock. No perfect positioning. etc.
It is odd that the hard part about making a game AI would be making an AI that isn't too competetive, but that's where we are.
Not really. It's just that we've devised a game that's difficult and interestin
Re: (Score:2)
This just in: Even in simplistic AAA games with bots, the AIs are better than human players, we have to dumb them down to keep the game fun.
Thats because 99% of AAA games are twitch games. No strategy involved, just reflexes. Games like COD, Halo, et al. really limit what the player can do and it really is the fastest mouse wins. So they have to limit the reaction time of the AI to what a human is capable of.
Now if you look at strategy games like Civ IV, you had to give the AI unfair advantages to put it on equal footing with human players. But turn based strategy games have a lot of fuzzy logic. So really an AI mastering COD is not news, we
Don't underestimate humans (Score:4, Funny)
No AI is a Wizard (Score:1)
I'd like to see AI take me on after a few quality bong hits while I rock the orbits and ramps of Black Knight 2000 with perfect captive ball shots or I slam-tilt a Star Trek:TNG table without losing my 50 cents. ...Or totally get 57 second of playtime and 36,000 points out of another 2 games and 6 lost balls after I hit nothing but the glass and scoreless bumpers while every shot goes down the side gutters or perfectly in the center of a triple flipper gauntlet totally getting screwed over by those damn mag
Re: (Score:3)
Damn, dude, you must be deaf, dumb, and blind.
Re: (Score:2)
Re: (Score:2)
Well, that would be the point. Things like driving a car and performing surgery need both (a) computing capability and (b) a real-world interface. It would seem that taking a pinball machine, a handful of linear motors, a camera, and some compute power would be a much, much more useful real-world learning exercise.
When playing an electronic game, the exact same input will produce the exact same result; but on a mechanical pinball game, there will always be a slight variation - which makes the whole thing fa
This guy did it in 2013... (Score:2)
So ... umm... (Score:2)
Now I can buy an AI so it can play the computer game I bought so I'd have more spare time?
Impressive accomplishment..but.. (Score:2)
..it's a computer figuring out how to beat a computer at a kinda simple game
The real world is a bit harder
Still..well done!
Paywall and some pdf rendering (Score:2)
Wonder why the editors let such bad sites and auto playing videos to be posted.
Re: (Score:2)
Which one is giving you issues? With noscript / firefox I had no problem with the science mag one. The nature one is really just a link to http://www.nature.com/nature/j... [nature.com]
It's cool but (Score:2)
It's actually really cool that this happened, so it's a shame that most of the reporting on it is sort of "correction bait". The fact that it does good at these games without watching human strategies is interesting, but computers have strategies that humans lack, due to their increased reaction time (random thing happened, I can respond by doing X -> the computer is several orders of magnitude superior at this for free) and increased calculation time (the trajectory is curve such and such -> your vi
Wait, not one classic arcade game in here? (Score:2)
It looks like it worked with Atari 2600 games, which are ports of classic arcade games. A nitpick, but about 30 seconds playing the 2600 version versus the arcade version will show you a ludicrous level of difference betwixt. I don't want to belittle the work, but calling 2600 games arcade games is like calling a motorcycle a semi truck. Words have meaning- in this case, "Atari 2600 games" or "classic games". NOT arcade games.
Re: (Score:2)
It's a misleading headline for sure. If you check the PDF it's pretty clear about the volume of 2600 games it works with, but they are def. not arcade games.
Surprising (Score:2)
Wow! For a generation, we knew that AIs are good at playing Global Nuclear War and now they tell us it's more like Super Mario Bros?
does it have a favorite? (Score:2)
which game is the AI's favorite?
Unfortunately... (Score:2)
Unfortunately, the experiment came to an abrupt end when they threw "ET: The Extra Terrestrial" at the AI, whereupon after an hour of trying different tactics the AI decided that the only way to win was to send a power surge through the system, frying the only working Atari 2600 the researchers could dig up.
This still classifies the AI as coming up with the best solution to the game ever implemented.
Yaz
Re: (Score:2)
Those early Atari 2600 ET adopters really had to stick their neck out with that game...
Comments are predictable... (Score:1)
You gotta love humans. Every time an AI starts to be able to do something that was previously only our domain, it's all "Meh, wake me when..." and "Yeah, but a computer still can't..."
Funny stuff.:)
Re: (Score:2)
Programming a computer to ace these games was possible in the 90s, maybe the 80s. The reason this is interesting is HOW it taught itself, and how many of the games it could get good at (many of the games it could not learn). Cheerleading AI research is nice, but this isn't an example of a computer entering a new domain, this is a research example of something that can solve other problems in the field- an engineering demo of sorts.
Re: (Score:1)
Obviously. The learning is the "something that was previously only our domain," not the playing, which is precisely what people are reacting so defensively to, and what I find funny.
Once AIs using ANN or whatever the ultimate technologies end up being can actually learn at a human level, it'll be "meh, wake me when it can appreciate a sunrise" or "Yeah, but a computer still can't fall in love!" My point is just that we move the bar in order to preserve our collective sense of being special snowflakes.
Pacman (Score:2)
Re: (Score:2)
That one is interesting because it took humans a ludicrous number of hours to figure out emergent patterns. It was trial and error (and the AI was deliberately not given an overly large amount of time on each game). Once the ninth key pattern was solved it became execution based. You could trivially code a machine to killscreen Pacman with no AI involved. Hell, you could probably do it with a very small Perl script- because humans already know the patterns that win.
It WOULD have been interesting to giv
press your luck (Score:1)
Well it's been 20 years and watson is still pressing his luck.
Re: (Score:2)
Re: (Score:2)
Reinforcement from... where? (Score:1)
Reminds me of a programming competition .... (Score:4, Interesting)
The best entries, however, didn't rely on AI, but on the fact that the RNG of the arcade game isn't random. Once the Asteroids-bot figured out the internal state of the RNG, it could basically use hyperspace to make targetted jumps (and never one that lead to the destruction of the ship), shoot at asteroids that haven't appeared yet and various other tricks. It was very impressive to watch one of these bots in action.
It did well for about 50% of the tested games (Score:2)
Of the 49 games tested there were about half which it did not do as well as a human player. They rated the performance of the AI against random play which equals 0 and a fairly skilled human player at 100%. The games the DQN agent did poorly at were:
It would be interesting to compare the games it did well at Vs those it did poorly at. Unfor
Zork (Score:2)
Now let's have an AI beat Zork.
http://thcnet.net/zork/index.p... [thcnet.net]
Breaking News: Computer pattern recognition ... ? (Score:2)
The computer mostly figures out the game design jsut like human players, and couples that with super fast reaction time.
The latter is the only reason it can beat the best human players.
The bigger story is: what games did it suck at and why?
Re: (Score:2)
new strategy (Score:2)
But did it enjoy playing them? (Score:2)
Source Code (Score:2)
Here is a link to the source:
https://sites.google.com/a/dee... [google.com]
Re: (Score:2)
That one got solved by the physics guys a while back. http://en.wikipedia.org/wiki/T... [wikipedia.org]