Quake Bots Rock The Prefrontal Cortex

0x4B writes "Some researchers from Vanderbilt University have used id Software's Quake III Arena to test a model of the human prefrontal cortex. The model was injected into the control systems of a Quake Bot, allowing it to flexibly adapt to changing enemy characteristics. The bot was required to identify the vulnerability of its enemies to different types of weaponry through repeated combat trials. Not only are the bots busy shooting each other relentlessly, but you can catch the action by joining the battle as an observer. Source code and Quake virtual machines are available for download."

Caught

[JohnFluxx]

Sounds like they were caught playing quake during work hours, and had to make up an elaborate story :)

nah just kidding.. But from the /. blurb, it's not that cool if all they can learn is what weapon is best.

Re:Caught

[Hogwash McFly]

With my experience with bots from various FPS games on several platforms is that the bots autoswitch to the most powerful weapon and stick with it until they die and/or run out of ammo.

The result? A close quarter battle where the bot is trying to snipe you from four feet away when he has a shotgun in his inventory. Or a map with copious amounts of one weapon becoming a single weapon map (Rocketfest, Sniperfest etc.)

If a bot can select an appropriate weapon based on a multiple factors than it serves to add to more realistic bot gameplay. Have I just popped in to a dead end? Then the bot should bounce grenades into my little enclosure. Am I clearly running for the flag? Then the bot should line the corridor with bio sludge to hinder my efforts. Am I flying through the air from a rocket jump? Then the bot should try and place a rocket at my landing spot as well as/instead of trying to continually fire rails at me.

Re:Caught

[ddd2k]

Actually quake bots are programmed to have weapon different preferences in their individual characteristics. the nightmare version of xaero has 99.9999999% accuracy with the railgun and hits u anytime anywhere any distance.

Re:Caught

[HTPC Junky]

I think this is a great step towards Agent Smith. Normal game AI is just scripted and predictable (boring). If this makes it harder to tell a person from a bot, then bring it on. I would love to see Half Life 2 with 32 internet players, and another 32 learning bots in the mix. I wonder how long before the bots consistency beat the real people!

Re:Caught

[Hogwash McFly]

Imagine if, in the near future, as well as a testing period, FPS developing houses have a 'learning period'. In this time there would be extensive testing of all maps/modes/mutators with just bot games. Over, say a one week period, the bots are left to their own devices, fragging 24/7, developing their algorithms on a daily basis as to become more and more adapted until at the end you, hopefully, have a very intelligent bot.

All you need are a little 'human' boundaries - +/- percentage of aiming error, +/- path deviation distance etc. defined at the start and you're set.

bots don't piss about or be irrational

[DrSkwid]

they day the bot switches to grapple and follows PCs round laughing at their hopelessness all the while grappling them to death I'll know their finished.

Or shooting someone with shotgun cos they'll fall backwards into the lava.

Or camp under the stairs when their bored

etc.etc.etc.

Re:Caught

[Devir]

that would be genetic algorithms. The menioned bots use what I believe is Neural network modeling specifically the frontal lobes of the brain. If genetic, neural and fuzzy algorithms could be merged into one bot you'd have one hell of a killing machine. of course you'd need some super computer to run them. Unless you only used the genetic algorithms for in house creation.

Re:Caught

[Soul-Burn666]

Not only that, they were caught using Aimbots!

Re:Caught

[tamer*f]

I would agree that a bot with only the ability to determine the best fighting weapon is not entirely impressive. Our main goal was to enhance the model we were working with so that it could function in a rich environment like QIII. What's cool is that the bot using this model was able to not only learn a strategy for weapon selection through practice, but then discard its strategy and quickly adapt when we changed the rules on it.

For instance, the experiment starts and our learning bot figures out that i

Output...

[wowbagger]

The output from this mod, after it has been running a while:

I frag, therefor I am.

How long until Valve adds this to HL2?

Perhaps this is what is holding up Duke Nukem Forever?

slow learners?

[den_erpel]

I just tried it for about 5 minutes (seems to me that this should be long enough for beasic learning), but the bots (with Quad colours) just seem to run around and do not react at all, no mather how many slugs I shoot at them.

I tried it with various settings of difficulty :( Am I mssing some README?

Re:slow learners?

[den_erpel]

It is getting better, I just discovered an magic weapon, think shotgun and machine gun mixed :) combined with god mode :P

the shots are on
, 1 [chello.be], 2 [chello.be], 3 [chello.be], 4 [chello.be], 5 [chello.be], 6 [chello.be], 7 [chello.be], 8 [chello.be], 9 [chello.be], 10 [chello.be]
Check out the waited shotgun shells :)

Re:slow learners?

[JBv]

Just a suggestion to make tthe pics clearer:

\seta r_vertexlight 1
\seta r_drawgun "0"
\seta com_blood "0"
\seta cg_brassTime "0"
\seta cg_drawCrosshair "5"
\seta cg_crosshairsize "24"
\seta cg_marks "0"
\seta cg_gibs "0"
\seta cg_blood "0"

RE: I didn't read the article, but I ran the mod

[monkeyserver.com]

Ya, RTFM, the point of this isn't to make them good at kicking your butt. This bot is supposed to be pitted against two special dummy bots, name one and two. Then it can react to the bots chaning their sheild. blah blah blah, it's actually a very cool use of nueral nets, but you wouldn't know that unless you read it :)

Re: I didn't read the article, but I ran the mod

[den_erpel]

Yeah, I figured that one out already after reading the sites.

I was a bit more enthousiastic, but now I think it's basically a first try, lacking an implementation, not really /. material if you ask me.

Sigh, again one of those decent ideas, but once again ideas do not implemented by themselves.

Move along, nothing to see.

Re: I didn't read the article, but I ran the mod

[tamer*f]

Hi dan,

I was wondering what gave you that impression about our mod. Did you run it correctly? Please refer to my post about seting up the experiment. I realize now that I should have put clearer instructions up on my site.

-Tamer

Re: I didn't read the article, but I ran the mod

[den_erpel]

Well basically I read the article with interest but messed up in one particular point: I used /addbot instead of /addannbot. k. I only noticed this after reading the article again this morning.

so /idscenario /addannbot

seems to do the job nicely :)

I am not that much interested in the gaming applications, rather than the experimental setup in itself. Basically a hobby from past research.

Being more verbose in the output would really be interesting.

Do you plan to put serveral ANN bots opposing each other?

reward

[tamer*f]

One of the things that made our task hard was the unpredictability of the Quake world. In other words, when the ANN bot misses a shot completely it is unable to get any useful info from it, potentially confusing it. The situation where it shoots the right gun at its enemy and hits the other bot sometimes arose as well. This made locking on to the right dimension difficult for the model because of this unreliable reward schedule. If we gave too much reward for a hit then the ANN bot could get stuck on so

Re:reward

[den_erpel]

Nice, it this 'mod' also available, because I find this one actually more interesting than the one you posted.

I think I would be able to spend quite some time to see these bots evolving :)

Not Close Enough

[Acts of Attrition]

An interesting development, but if they really want to replace human players they still need algorithms for name-calling, complaining and cheating.

Re:Not Close Enough

[Hogwash McFly]

I know that was intended humourously, but have you ever played Counter Strike with POD Bot? Damn that gets annoying, with all of the random name calling and complaining it's just way too OTT. I mean, I know that I'm not playing with real players so why try and fool me into thinking that a real person just told me 'great shot Hogwash!' when I'm just camping behind a crate?

Re:Not Close Enough

[Alpha_Nerd]

They do!!

I was playing with some random CS bot a few years ago... And all of the sudden it started playing 10x better and started saying things I had never seen it say before. For a few minutes I thought maybe it had evolved lol.

AI programming

[one9nine]

Can anyone comment on what types of AI algorithms they use to do something like this (neural networks, markov, etc.) or know of any good resources on the net?

Re:AI programming

[tamer*f]

The model of PFC that we used is a simple backpropagation artificial neural network with some modifications. Recurrent connections to layers with special activation weights enable the working memory and cognitive control functionality of the model. It would be a little much to clearly describe it here, but if you are interested you can check out http://psych.colorado.edu/~oreilly/pubs-abstr.htm l #01_id_ed -the original paper describing the model. I've made an effort to describe it in simpler terms on

That reminds me of the time...

[Smid]

When we left our clan Unreal Tournament server going for a week with godlike bots with no timelimit and forgot about them...

Came in for a visit, and it was loque, tamara and another versus poor old Desloch by himself. 769 caps to 212.

Evened the teams a bit and Deslochs team had picked up another 400 caps a day later. What a fighter, eh?

An ending to the story

[PD]

"And that was how I spent my childhood", said the T1000.

help!

[Polo]

We must create an Arnold bot to defend ourselves!!!

How to run the mod

[tamer*f]

It seems like it is a little unclear to some people how to correctly run our mod (largely due to me leaving out significant details on the download page). It is intended as an experiment involving one learning bot utilizing the PFC model and two special dummy bots. You need to make sure you run the game using the map I made so that the bots have constant interaction. Using the command line listed on the download page, start the game, and then switch yourself over to be an observer. Run the following console command: /idscenario

Now you have some very colorful dummy bots with unoriginal names running around doing nothing. But it gets better....

Use the console command /addannbot just like you would /addbot to put a learning bot in the mix. For best results, use a bot that doesnt suck. Now switch over to the learning bot's first person view to see the status of its Neural network and PFC layers changing as a result of its perception. Igor, its aliiive.

Now, there is no damage or death in the mod because we didnt want this to complicate the experiment. What you should see is a blue icon appear above an enemy that is hit. A deflected shot will bounce off like it hit the invulnerability sphere. When the bot hits, you will notice the little white box at the top of the network status overlay (upper right corner of the screen) go solid. This signals that the bot got a reward.

Directly under that are two yellow boxes, these represent how much the bot wants to choose each weapon (full is highest). Once the bot learns something you will notice these switching dramatically in response to the characteristics of its enemy. The bottom row of red boxes shows the characteristics of the current enemy (shield color, gun color, position, ID).

Now with all this information the bot tries to figure out what about its opponent is important in deciding how to kill it. The top row of yellow boxes at the left of the screen encodes what dimension the bot is considering, shield color, gun color, location, or ID(name). When the bot picks the right dimension, it can reliably slam its opponent with the right beam color. When it chooses the wrong dimension, it performs miserably until it gives up and explores something else.

Our experiment is set up such that the first correct dimension is shield color. After the bot figures it out the experiment will autoswitch to the ID dimension. When this happens you will see a message appear at the top of the screen in red. When its behaving well, the bot will catch on quickly.

Thanks for checking out the mod, and sorry about being late with this info. If you've got questions a lot is explained in the code walkthrough on the site, otherwise just ask me here. cheers,

-Tamer