New AI Is Capable of Beating Humans At Doom (denofgeek.com) 170
An anonymous reader quotes a report from Den of Geek UK: Two students at Carnegie Mellon University have designed an artificial intelligence program that is capable of beating human players in a deathmatch game of 1993's Doom. Guillaume Lample and Devendra Singh Chaplot spent four months developing a program capable of playing first-person shooter games. The program made its debut at VizDoom (an AI competition that centered around the classic shooter) where it took second place despite the fact that their creation managed to beat human participants. That's not the impressive part about this program, however. No, what's really impressive is how the AI learns to play. The creator's full write-up on the program (which is available here) notes that their AI "allows developing bots that play the game using the screen buffer." What that means is that the program learns by interpreting what is happening on the screen as opposed to following a pre-set series of command instructions alone. In other words, this AI learns to play in exactly the same way a human player learns to play. This theory has been explored practically before, but Doom is arguably the most complicated game a program fueled by that concept has been able to succeed at. The AI's creators have already confirmed that they will be moving on to Quake, which will be a much more interesting test of this technologies capabilities given that Quake presents a much more complex 3D environment.
I can beat AI with a hammer. (Score:2)
Bring on your pesky AI. What I don't break with the hammer, I will break with my urine. Bzzzt!!!
Re:I can beat AI with a hammer. (Score:4, Funny)
I will break with my urine. Bzzzt!!!
If that's the sound you make when you pee, you should see a doctor.
Re: (Score:2)
It is the sound of electronics shorting out.
Paying a doctor to look at my dick is a dumbass idea.
Re:We already have that! (Score:5, Funny)
Yeah, but does the AI live in its parents basement and pee in empty Mountain Dew bottles ? I think not!
AI may win at DOOM, but we're superior LOSERS!
So there...
Doom, the game where you can't look up or down. (Score:2, Funny)
Well I for one welcome our new 2d overlords..
Re: (Score:2)
Pioneered an industry? The game you're referring to is called Pong. The industry was in its third decade by the time Doom came along.
Kids these days...
Re: (Score:1)
as great of a game that was, wolf3d was household name, not blake stone.
Re:The screen buffer? Really? (Score:5, Insightful)
You totally miss the point: it's trivial to write a perfect bot that hooks into the game's internals and always wins. It's difficult, and more generally applicable, to make a bot that learns to play by watching only the same info the human players get: the screen buffer.
Re:The screen buffer? Really? Consider... (Score:1)
The scary part is not an AI bot beating you in a game.
The very scary part is an AI bot (with physical presence and simple vision) beating, maiming, or killing you into submission when you and [hundreds, thousands, millions] of your fellow humans rebel against an oppressive, intrusive state that uses these bot to control the "rabble", justified by "Law and Order".
- Leonard
Re: (Score:1)
Good part is - we short circuit the entertainment industry like this means we win some private time we can do something else....
Re: (Score:2)
Re: (Score:3)
It's difficult, and more generally applicable, to make a bot that learns to play by watching only the same info the human players get
That's not what's happening.
We've seen similar bullshit headlines like this before that imply that the program learned to play the game just from watching the screen. That couldn't be farther from the truth.
Prepare yourself for some serious disappointment and read the paper.
Re: (Score:2)
That's not what's happening.
The summary, article, and paper all say you are wrong.
Re: (Score:2)
Looks like someone didn't read the paper. It's free, so it's not like it'll cost you anything to read it.
Here's a helpful hint: When you hear than an AI "taught itself" anything, it's guaranteed to be bullshit for the foreseeable future. Simply things like determining success criteria are far-beyond what so-called AI can actually do that it's laughable to say it "taught itself" anything.
They trained a NN just like we've been training NN's for ages. It's about as interesting as an undergrads NN project.
Re: (Score:3)
How is this not using the screen buffer?
Re: (Score:3)
Who said it didn't use the screen buffer?
My point was that it didn't learn to play the game exclusively using feedback from the screen buffer, like the magical thinkers here seem to believe.
Re: (Score:2)
Ahh, my apologies for the previous post.
You've posted your lies 7 times in this thread so are clearly just purposefully trolling.
Go ahead and disregard all the facts and don't bother replying to me.
Re: (Score:2)
What about my post was false?
This toy did not "teach itself" to play the game using only feedback from the screen buffer. That is a very simple and obvious fact. How you can believe otherwise is astonishing. Read the damn paper.
I know. No one likes to have their silly fantasies shattered by the cold light of reality, but enough is enough. Face facts, read the paper, and get over it. The last thing we need is another technology-based religion like the "less wrong" group or the Kurzweil cultists..
Re: (Score:2)
From the actual paper: "Game Settings: The game’s state was represented by
a 120 × 45 3-channel RGB image, health points and the
current tick number (within the episode). Additionally, a kind
of memory was implemented by making the agent use 4 last
states as the neural network’s input. The nonvisual inputs
(health, ammo) were fed directly to the first fully-connected
layer."
So yes, they gave it two integers from the internal workings of the game, solely because the player gets those values from t
Re: (Score:3, Insightful)
Re: (Score:2)
You should buy that.
Until there's a link between brain and game to manipulate the controls, the Script has an unfair advantage. I have to use a keyboard, the script doesn't.
Re: (Score:2)
"I don't buy that at all... the computer in this case is still learning through visual feedback only."
Instantaneous visual feedback. Humans have serious input and output lag in comparison. Give the bot the exact same latency we've got and see how it does.
Don't most games do this... (Score:2)
It's funny to see some mainstream press outlets freak about about this like it was Skynet when three is already so much anti-layer going on in games already...
Many games have creatures that hunt players, usually programmed with a somewhat limited view of the entire internal game world just as this AI has only the screen to understand where objects are.
So I guess I'm missing what is really new about this, I assume some in-game AI in a few games somewhere has already had behavior programmed by some kind of le
Re:Don't most games do this... (Score:5, Informative)
The point is that the AI learned to play the game from only screen data. No maps, no preset strategy, just visual data. So, it has to learn to recognize threats and obstacles, and what to do when it does.
Beating humans is a good test, because humans are good at exploiting patterns, so shortcuts like always taking a fixed route wouldn't work for long.
Re: (Score:1)
The point is that the AI learned to play the game from only screen data.
That is a very false statement.
Re: (Score:2)
Or you could actually read the article, I know I know its hard so I did it for you. From the abstract of the journal article;-
Re: (Score:3, Informative)
From TFP
b) Game Settings: A state was represented by the most
recent frame, which was a 60 × 45 3-channel RGB image.
The number of skipped frames is controlled by the skipcount
parameter. We experimented with skipcounts of 0-7, 10, 15,
20, 25, 30, 35 and 40. It is important to note that the agent
repeats the last decision on the skipped frames.
and
b) Game Settings: The game’s state was represented by
a 120 × 45 3-channel RGB image, health points and the
current tick number (within the episode). Additionally, a kind
of memory was implemented by making the agent use 4 last
states as the neural network’s input. The nonvisual inputs
(health, ammo) were fed directly to the first fully-connected
layer. Skipcount of 10 was used.
The only thing they gave the AI for the experiments were the screen image, the time, and in the second case, the health.
Re: (Score:2)
Yes, it plays with only screen data. It did not learn how to play using only that data. Just like a zillion similar NN projects before it.
Sorry if this hurts your religious beliefs, but reality is indifferent to your fantasies.
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Wishful thinking from the futurists?
The silly belief here is that the program "taught itself" to play using only feedback from the display buffer. That is so obviously untrue that I can't understand how anyone could believe it, let alone repeat it!
Read the paper. It's no different than the average undergrads NN project.
Re: (Score:2)
You've yet to explain your point. It learned to play the game, which is in fact somewhat like the average undergrads NN project (perhaps a bit more elaborate in scope). And ...?
Re: (Score:2)
It's not difficult. The implication here, from the article and summary, is that the program learned to play the game using only feedback from the display buffer. That is, quite obviously, false.
As I pointed out earlier, it did not, and can not, determine success criteria. That is the assumption you see endlessly here, implicit in absurd statements like "the computer in this case is still learning through visual feedback only", "The point is that the AI learned to play the game from only screen data. No m
Re: (Score:3)
You seem to be slicing something very fine that seems of minor importance, unless I'm missing something. When a human learns to play Doom, he starts knowing the success criteria, he knows upfront that the basic tasks are "maze explorer" and "move and shoot", and what the health bar is. Yet we still say he "learns to play Doom".
From what I read, the bot learned the map, learned to move and shoot in an effective way, and had only the screen buffer and a health number as an interface with the game - not the
Re: (Score:1)
When a human learns to play Doom, he starts knowing the success criteria
I couldn't disagree more. New players can, and do, learn how to play from either seeing someone play or by interacting with the game. They don't need to be told any details. They can learn how to play exclusively though visual (or visual and auditory) feedback. They can learn how keys and doors work, that they can kill monsters, and that they progress by finding the exit in each level all on their own. On health, they don't need to be told that taking less damage is better than taking more damage.
They
Re: (Score:2)
I guess I still agree with like "the point is that the AI learned to play the game from only screen data. No maps, no preset strategy, just visual data". We're arguing about what "learned to play the game" means. You seem to be objecting that "that part the computer can't do yet, that's the important part [wikipedia.org]", which people have been saying about AI research for 50 years of progress.
Re: (Score:2)
The fact remains that statements like "the AI learned to play the game from only screen data" are completely false. It had far more than data from the screen buffer available to it.
We're not arguing over what "learned to play the game" means, we're arguing over the claim that the program learned having access only to the screen buffer, which is completely false.
Why does that matter? Because the uninteresting part, they trained a NN by traditional means, isn't what people are claiming. They're claiming th
Re: (Score:3)
Human newbies have a ton of context to determine what's success in the game. I think you're the one who seems to expect a Doom bot to have solved all of AI forever.
This appears to be a significant improvement over something like Breakout or even Mario, and just because it was partially supervised, doesn't make it less of a progress.
Re: (Score:2)
I think you're the one who seems to expect a Doom bot to have solved all of AI forever.
No, I'm saying that the article and a lot of readers seem to believe that a significant problem has been solved, which, obviously, has not been solved.
You seem to want to believe there is far more to this project than is actually there. Wishful thinking is fine, but don't pretend that it's reality.
Re: (Score:2)
The (reasonable) claim isn't that the bot "learn[ed] to play solely from feedback from the display buffer", unless you add "exactly like a human would". A human would bring a lot of context before he started "learning". Sure, we can slice it very fine about whether a human could figure out what a health bar was, eventually, without knowing ahead of time, but then, perhaps that NN would too given a longer training time.
The main thing is, if we go back to when Doom was new, the human already knew that dyin
Re: (Score:1)
We're not going to make any progress here as long as you refuse to address the only claim I've made. To be honest, at this point, I can only assume you're trolling.
Re:Don't most games do this... (Score:4, Informative)
Ehh, a first-person shooter is not really all that great a test for AI. Being a videogame programmer, and one who has programmed his share of AI in commercial videogames, the trick is not to create unbeatable AI, but to create an interesting experience for the player. Granted, we use a lot of internal data structures to assist the AI (like for navigation), and we obviously approach things from a completely different angle, but we also have to drastically handicap the AI's responsiveness and aim in shooter-type games.
Remember, it's trivial for a computer to paint a bulls-eye between your eyes from 500 yards out with a sniper rifle no matter how you're moving or hiding. It's still reasonably easy even with true projectiles, as the AI can calculate perfect flight trajectories so that rocket will precisely intercept a moving target. An AI can't get disoriented, or confused, and has near-instant reflexes that no human can match.
One trick I've used for shooter bots is to incorporate virtual springs attached to the bot's targets, helping to throw off their aim according to how the target is moving. You can also dynamically adjust the target spring tension or based on other factors, like difficulty scaling, whether the AI agent is running, jumping (throwing off his own aim), etc. That sort of thing, along with adding blind spots, artificial reaction times, intentional mistakes, and so on, are the things you have to do to keep the bots from kicking the crap out of human players just because they could instantly headshot you from across the map otherwise.
Don't get me wrong... this is a neat little project. But beating humans in a shooter where fast reflexes and perfect aim dominate isn't really the end-all and be-all of AI tests, because our strengths and weakness are in different areas than for computer-based opponents.
Re: (Score:2)
You've missed the point I think. This isn't an aimbot.
This is a neural net that needed to learn on its own what an enemy looked like on the screen, and what obstacles were, and how to move around them, and so on. The basic "strategy" logic of the bot was fairly simple: a maze explorer, plus an aimbot. The interesting part was doing that given only the screen buffer to work with (well, they cheated a bit by giving the health explicitly).
Re: (Score:1)
The novelty is not in the AI itself but rather that it's only being fed the exact same video data that a human player sees. Essentially applying the same technology that a self-driving car uses, mashed up with a neural net and instructed to learn to play.
Re: (Score:2)
Re: (Score:3)
Right. So if you think you're going to "out-draw" the AI, you're probably SOL.
Which means you have to out-think it. Snipe it, staying out of it's vision.
Re: (Score:2)
Which is what makes it an interesting test of an AI. It's not like aimbots are new or anything, but an AI that humans fail to outsmart is something. Of course, the devil is in the details.
Re: Don't most games do this... (Score:3)
Re: (Score:2)
Re: (Score:3)
There's a pretty big difference between a game AI (which is fed machine-centric game state information, and has an extensive pre-programmed ruleset) adapting marginally to a player's actions, vs learning to play (and master) the entire game via screen inspection.
Re:Don't most games do this... (Score:5, Informative)
In-game bots may be operating on a limited view, but they're operating on actual hard data in basically machine-usable form. What's impressive about this is that it learns from what's on the screen -- distances, obstacles, paths, its location are all learned from visual input.
What I'm curious about is how adaptable their visual learning system is or whether it's extremely Doom specific. I'd also be curious at how long it took to learn to play. I'd also be curious what the learning curve was -- linear, non-linear, flat, steep or what.
Re: (Score:2)
Re: (Score:1)
. What's impressive about this is that it learns from what's on the screen -- distances, obstacles, paths, its location are all learned from visual input.
Wow, no. The bot has access to the depth buffer. One of many, many, reasons why reality doesn't match up with the expectations set by the headline and summary. Read the paper. This is about as impressive as the average undergrad NN project.
Re: (Score:1)
> The bot has access to the depth buffer. /have/ a depth buffer. And the paper is quite explicit that the only input is an RGB framebuffer.
Doom doesn't
Re: (Score:2)
Re:Don't most games do this... (Score:4, Insightful)
Modern re-implementations of Doom and Wolfenstein have a depth buffer. The original do not.
Doom used BSPs to sort walls from front to back and a scanline like algorithm that guaranteed no overdraw, so there is no use for a depth buffer. Sprites were sorted from back to front, and occlusion with other sprites was handled by painter's algorithm, and occlusion with walls was done by using a list of drawn walls and direct intersection testing. Again, no use for a depth buffer.
The algorithms in the original Doom wasn't perfect, and there are some severe artifacts to find if you know where to look, especially if sprites where near a step. Also, re-implementing Doom in something with a depth buffer like OpenGL either requires modification to how sprites are rendered, or you will get a visual difference from the original game. In Doom, the lack of clipping of sprites against the floor and ceiling is usually easy to handle by just shifting of sprites vertically, for example. But some other games made with the Doom engine depended more heavily on the lack of such clipping in the perspective used by the artists creating the sprites, and it can take some effort to get OpenGL to not clip things clip when that weren't clipped in the original game.
Re: (Score:2)
Nonsense. In Wolfensten, for example, a 1D depth buffer is NECESSARY to paint partially eclipsed sprites, and to avoid painted completely eclipsed sprites.
This is how it works: On the first pass, a ray is cast for each vertical column on the display. When a wall is encountered, the distance to the wall and the position of the ray on the way is determined and used to draw a vertically-centered column scaled by distance. The distance is stored in a buffer the width of the screen. (A Depth Buffer!)
On the s
Re: (Score:2)
So I guess I'm missing what is really new about this
Looks like.
no video? (Score:4, Insightful)
this is a video game, after all...
Nice? (Score:3)
I would have thought AI had already beaten us at those games long before beating us at chess or go. Note sure if I am supposed to feel good as a human or bad as a programmer now.
Re: (Score:1)
The novel part isn't that it can beat humans. As previously observed, that is a simple job for an aim-bot combined with some predefined, map-specific movement patterns. The novel part here is that it taught itself how.
Re: (Score:2)
It did no such thing.
It's not about the performance (Score:2)
They are called... (Score:2)
So... in a few years... (Score:5, Insightful)
Can I rig Call of Duty with an AI auto-pilot plug-in, and just sit back and watch it steam-roll over all the sucker humans in the game? If I play an online game like Overwatch and get smeered over and over by an opponent(s) with perfect aim and lightning-quick moves, will I just assume someone's introduced a bot into the game and I'm wasting my time with my hopelessly inferior carbon-based reflexes? Gaming may need its own version of the Butlerian Jihad.
Re: (Score:3)
Re: (Score:2)
It sounds pointless, but CSGO has a real world economy so this could be used for real financial gain.
Re: (Score:1)
FOOLS! (Score:1)
Do you want Skynet?! Because this is how you get Skynet!
Not Impressed (Score:3)
Until's middle finger hurts from pressing the "W" for five hours straight, I will not be impressed. (yes I did this in 1994).
Re: (Score:3, Insightful)
Sorry for the 20-year late message.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
I remember computer games that used WASD for movement in the 1980's.
Ramifactions for the Future of Gaming (Score:5, Insightful)
It will be interesting to see how future games develop to keep them fun for humans in an AI-filled world.
Imagine your AI setup gets to the point where it truly has the same input, not needing to be directly fed the screenbuffer but can use a camera pointed at your monitor. Suddenly current anti-cheating technologies mean nothing, and enough people using these would quick ruin a game.
Re: (Score:2)
Reminds me of an xkcd about reputation-based spam-blocking [xkcd.com]. Seems we might have a similar situation here, except that we end up with everyone benefitting from being able to play against the most enjoyable opponents all the time.
Re: (Score:2)
Re: (Score:2)
Suddenly current anti-cheating technologies mean nothing, and enough people using these would quick ruin a game.
Contrariwise, imagine a world where you can play in your computer against any number of AI opponents, regulated to the level that makes the game interesting to you. Then you don't need other people and cheating becomes meaningless, as it should be in a game.
Re: (Score:2)
Re: (Score:2)
So you'll end up with bots as teammates and worse players as opponents.
Marvel's way ahead of you.... (Score:2)
They've had Doombots [wikipedia.org] for years.
Good thing it didn't go up against me (Score:2)
Impressive, but flawed (Score:3)
Doom may seem simple compared to Quake, at least superficially, but Doom features the BFG 9000 which a good player can do some fairly impressive things with, that would be VERY hard to deduce from simply observing. How the BFG worked wasn't really worked out in full detail until the source code was released. The BFG9000 is probably one of the most complex FPS weapons in any mainstream game. Then there are techniques like wall running, bumping, silent BFG shots etc. Knowing about these and when they are of use, can give a player a huge edge. Can the bot discover, use and master this? Such techniques are vital on the most common deathmatch maps, map01 and 07 in Doom 2.
Doom deathmatch can also be played in altdeath mode, typically map11 or maybe map16 are used for this type of play. This introduces many new skills, and downplays other. It is a rather different experience. Does the bot handle this? Navigating the 3d space of map11 is a lot more complicated than map07, which is basically flat. Figuring out the map, teleporters, secret doors, trigger lines that activate elevators, etc is pretty complicated stuff.
Given phrases like "Their agent, he said, “was ducking most of the time and thus was hard to hit.” I suspect a good human player would outskill the bots here. From the ViZDOOM paper (https://arxiv.org/abs/1605.02097) "we test the environment by trying to learn bots for two scenarios: a basic move-and-shoot task and a more complex maze-navigation problem."
When it comes to singleplayer, I would love to see bots play better than Henning in his 30nm run in 29:39, https://www.youtube.com/watch?... [youtube.com]
There was a nice AI bot mod in Quake C (Score:5, Interesting)
All the way back in the original Quake there was a really nice learning AI written in quake C. One version allowed you to add practice bots to work on your deathmatch strategy.
Similar to the AI described in this article, the AI in this mod was ignorant of the map and had no preset patterns. It learned by doing. So as you began playing against them they were easy kills in the early rounds. They'd often just stand there and get shot. And they couldn't hit the broad side of a barn.
But they learned the map. And they learned your moves. And within a few rounds you'd be lucky to stay alive long. And finally they would learn enough to get you every time. They'd know which direction you were going to dodge before you did. And they kept track of every resource in the game and all of the respawn times, so they'd deny you any ammo or health by timing their movements perfectly to collect all spawns instantly.
It was very cool.
Then the guy who wrote it used his AI to replace the original game AI for all of the enemies. Wow. It made the game into an entirely different experience.
After about a half-level, the enemies would learn to avoid you, go out and recruit all the bad guys from the level and return in force. After a couple of more levels they'd learn to ambush, flank and surround you. They'd team up their fire, so you'd dodge a fireball to the left and strafe right into another fireball.
It was really interesting, but ultimately unplayable. It really gave me an appreciation of the level of "balancing" that goes into creating a proper game AI. It certainly isn't about the same thing as making a chess AI that can beat Kasparov. It requires a great deal of work to make the enemy realistic and interesting and difficult but ultimately beatable.
Re: (Score:2)
All the way back in the original Quake there was a really nice learning AI written in quake C...But they learned the map. And they learned your moves. And within a few rounds you'd be lucky to stay alive long. And finally they would learn enough to get you every time. They'd know which direction you were going to dodge before you did. And they kept track of every resource in the game and all of the respawn times, so they'd deny you any ammo or health by timing their movements perfectly to collect all spawns instantly.
It was very cool.
I used to play quite A_LOT of Quake and Quake 2 and 3 and don't recall this. Bots have always been a split between extremely useless or extremely impossible (ie they know where you are and head-shot you before they've seen you). Even now in the latest games the bots are easily identifiable for the same reasons, either too crap or impossibly good.
Re: (Score:2)
I used to play a ton of Quake and Quake 2. I *think* Cytotoxic is referring to Eraser bot. As noted here [quakewiki.net], the bot will learn maps it has never seen before. Now, I don't remember ever seeing any documentation about the bots learning your play style or anything, but I do remember most of the rest of what Cytotoxic said.
A Quake 2 version existed as well, so a friend and I used these bots in Quake 2 to test custom levels. At first, some would run around, not picking up much. Others would just sit still unt
Hah, it can't beat humans at quake unless... (Score:2)
Re: (Score:2)
one small step for man... one giant leap for .... (Score:2)
Re: (Score:2)
They claim the AI Doom marine ducked ...WTF? (Score:2)
The last second to last sentence of the article says, "...the developers claim that their AI won one of the competition games by learning to duck and therefore making itself much harder to hit." What version of Doom are they playing? As a teenage I played countless hours of Doom 1 & 2 and I don't remember a duck/crouch button.
Re: (Score:2)
Being able to beat a human isn't a big deal. Being able to do so while using the exact same inputs (key presses) and outputs (a picture of the screen) as a human is a big deal. Doom is definitely a simplified problem set (a given sprite only ever varies in scale and X/Y position), but it's still an impressive feat of machine vision and machine learning.
Re: (Score:2)
In what way? The entire point of this project is that it relies on the screen buffer, hence the name "VizDoom".
Re: (Score:1)
Pointing a camera at the screen is nothing like how human vision works, as human vision is a complicated mess of dynamic gain and feedback, not a simple rectangular array of values. A mechanically controlled mouse and keyboard is nothing like a human, as you can easily move it faster and with more precision than any human ever could. Adding a camera and mechanical controls could easily be simulated by just adding lag to outputs and noise to inputs, but how lag affects a control system and noise affects co
Re: (Score:1)
If you actually read the paper, you'd know that the blurb is complete fiction.
Re: (Score:2)
They took the same reinforcement learning techniques from the Atria 2600 AI and applied them to Doom. It worked. Next they plan to do it on Quake. Let me give you a hint for the future: It'll work.
It is still noteworthy, since the Atari visual frames are built up from sprites of limited size that are very convenient for a convolutional model: once it has learned to detect the sprites, it has an accurate representation of the location of the relevant objects in the scene. In doom, since it is a 3D scene, there is no such one-to-one correspondence between object identities and visual appearance. The fact that the method still works is not at all trivial.