BotPrize — A Turing Test For Bots

Philip Hingston writes "Computers can't play like people — yet. An unusual kind of computer game bot-programming contest has just been held in Perth, Australia, as part of the IEEE Symposium on Computational Intelligence and Games. The contest was not about programming the bot that plays the best. The aim was to see if a bot could convince another player that it was actually a human player. Game Development Studio 2K Australia (creator of BioShock) provided $7,000 cash plus a trip to their studio in Canberra for anyone who could create a bot to pass this 'Turing Test for Bots.' People like to play against opponents who are like themselves — opponents with personality, who can surprise, who sometimes make mistakes, yet don't robotically make the same mistakes over and over. Computers are superbly fast and accurate at playing games, but can they be programmed to be more fun to play — to play like you and me?" Read on for the rest of Philip's thoughts.

Philip continues, "Teams from Australia, the Czech Republic, the United States, Japan and Singapore competed in the final. Competitors created bots to play a specially modified Unreal Tournament 2004 Death Match. Expert judges then tried to tell whether they were playing a bot or a human, just from their observation of the way they played the game. Judges included AI experts, a game development executive, game developers, as well as an expert human player. The result? The winning team AMIS, from Charles University in Prague, managed to fool 2 out of the 5 expert judges, and achieved an average 'human-ness rating' of 2.4 out of 4. All the human players were judged more human than the bots overall, but the judges were fooled often enough to suggest that in next year's contest, some bots may be able to pass the test by fooling 4 out of 5 judges. AMIS won $2,000 cash plus an all expenses paid trip to 2K's Canberra studio. You can check out the full results and competition videos, and try an online video quiz that lets you judge for yourself."

Fun bots isn't all about realistic opponents

[LaurensVH]

The fact that you're actually playing a human is a big factor too. Fast connections and low ping times aren't the only reason LAN parties were successful -- sometimes you just want to rub it in.

Re:Fun bots isn't all about realistic opponents

[DiLLeMaN]

That's the whole point: getting pwnd without some smug human to rub it in. Much more fun to lose that way.

But how to tell the bots from humans?

[David Gerard]

"Good morning."
"STFU N00B"
"Er, what?"
"U R SO GAY LOLOLOLOL"
"Do you talk like this to everyone?" "NO U"
"Sod this, I'm off for a pint."
"IT'S OVER 9000!!"
"..."
"Fag."

How do you make a computer act stupid enough [today.com] to imitate actual humans?

Human Tendancies

[Archon-X]

As a casual gamer + AI observer, in my opinion the biggest / most obvious difference are human traits.
While this may sound obvious, let me elaborate:

- Traits are different to mistakes or intelligence. Mistakes are missing, shooting into walls, walking over edges, etc.
- Traits are: becoming too involved in a firefight, that you *know* you're going to lose, being so wound up on one enemy that you miss seeing others, hiding behind corners to wait for others to become injured, etc

Playing against humans has much more appeal than bots, because people are 'fun'. No bot is ever going to run at you with an axe ( or other lowest equivalent weapon ) when you've got the BFG - but humans will - and will often win with this tactic through sheer stupidity or blind chance.

I can only imagine programming human traits is a lot more difficult than 'standard' AI.

In the videos, I got most of the choices right by applying the question: Who is applying human behavioral patterns?

Re:Half life or Counter Strike...

[Fri13]

I remember those bots too, they were first ones maded for the Counter-Strike mod. Then they suddnly started to be bots again when they included the ultimate skills for them, like complete 180Â turn in one millisecond. So you just ended to situation that bots killed you before you even saw them behind corner what was situation with every human player on game itself.

TA Spring RTS engine (http://spring.clan-sy.com/) has few AI's to what are made to learn from human players. They just actually collect information about units, abilities etc. And then positions on the map where they got killed and then they avoid or attack more agressive to that position. So very limited AI but sometimes nice opponent for single player.

Hopefully people would start developing good open source AI's for different games, so every game developer would not need to build everything from scratch.
We could end up to AI's designed for RPG, FPS, RTS and simulations etc games. Design them to use plugins, so you can import wanted features from other areas AI's and configure it work correctly on different situations. Then somekidn nice player recorder what would calculate the human player movement and adapt it.

Re:Image-based bots

[im_thatoneguy]

The only exception to this would be a difference matte of the player.

I would 'render' out a patch around any visible opponent to see *how* visible they are.

If they're standing still in front of a tree trunk the average luminance at the border of the player and tree need to measured. If the player is instead a black SWAT player on a white snow field then his visibility would be increased.

Motion should also be multiplied.

Contrast * %of Player Visible * percieved velocity (If they're crouching and creeping at '100 yards' they'll be moving slower in the bot's view than if they were 2" in front of them.

You want to make sure that just because the player model is visible it doesn't mean the player would actually be visible to an opponent. I can't count how many times a black hooded enemy in a darkened window has sniped me.

Also a bot should have its sound perception nerfed. Just because it hears a set of footsteps doesn't mean it shouldn't be biased to stereo or at least 5 channel restrictions.

Re:Really Means Effective Artificial Intelligence?

[DreadPiratePizz]

Not going to happen. I realize that most people don't like to admit that technology has limitations, but the fact remains that the human brain processes information in both a representational, and non representational way, while computers are limited to representational data processing. First of all, a lot of human "knowledge" is embodied knowledge, solidified in the brain by a continuous feedback between it and our environment through means of our bodies. Computer don't have bodies, and can't have embodied knowledge. You cannot simply program a 'fake' body either, because the shaping of the neural pathways is cumulative and precisely dependent on the body. It would be like putting Roger Fedderer's brain in Rosie Odonell's body. I will guarantee you that he could not play tennis after that. He probably couldn't play with Pete Sampress' body either. Computer games are as much a motor skill as they are an intellectual exercise , so unless your program has access to a human body, it can never play like a human, because motor skills are developed through years of feedback between the environment and the body, and cannot be broken down into rule based axioms.