BotPrize — A Turing Test For Bots 79
Philip Hingston writes "Computers can't play like people — yet. An unusual kind of computer game bot-programming contest has just been held in Perth, Australia, as part of the IEEE Symposium on Computational Intelligence and Games. The contest was not about programming the bot that plays the best. The aim was to see if a bot could convince another player that it was actually a human player. Game Development Studio 2K Australia (creator of BioShock) provided $7,000 cash plus a trip to their studio in Canberra for anyone who could create a bot to pass this 'Turing Test for Bots.' People like to play against opponents who are like themselves — opponents with personality, who can surprise, who sometimes make mistakes, yet don't robotically make the same mistakes over and over. Computers are superbly fast and accurate at playing games, but can they be programmed to be more fun to play — to play like you and me?"
Read on for the rest of Philip's thoughts.
Philip continues, "Teams from Australia, the Czech Republic, the United States, Japan and Singapore competed in the final. Competitors created bots to play a specially modified Unreal Tournament 2004 Death Match. Expert judges then tried to tell whether they were playing a bot or a human, just from their observation of the way they played the game. Judges included AI experts, a game development executive, game developers, as well as an expert human player. The result? The winning team AMIS, from Charles University in Prague, managed to fool 2 out of the 5 expert judges, and achieved an average 'human-ness rating' of 2.4 out of 4. All the human players were judged more human than the bots overall, but the judges were fooled often enough to suggest that in next year's contest, some bots may be able to pass the test by fooling 4 out of 5 judges. AMIS won $2,000 cash plus an all expenses paid trip to 2K's Canberra studio. You can check out the full results and competition videos, and try an online video quiz that lets you judge for yourself."
Fun bots isn't all about realistic opponents (Score:4, Insightful)
Re:Fun bots isn't all about realistic opponents (Score:4, Insightful)
That's the whole point: getting pwnd without some smug human to rub it in. Much more fun to lose that way.
Re: (Score:2)
That's the whole point: getting pwnd without some smug human to rub it in. Much more fun to lose that way.
No, it's really not. Getting completely pwnd and never being close to winning all the time is frustrating whether it's a human opponent or computer controlled one. On the other hand, a close match is much more entertaining when there's someone rubbing it in. It's the motivation for another game.
Case in point, I remember playing x-wing alliance time trials against my college roommate. Playing that on your own gets boring quick. Beating the other person's record by a few milliseconds would cause trash ta
OK... (Score:1)
Sorry, but can't resist:
"... and second prize was TWO trips to Canberra."
*ducks*
Re: (Score:1, Funny)
The Sexdroid Turing Test. I volunteer.
Excellent. We were hoping for a volunteer.
The first Sexdroid competitor will the Reamatron 9000.
Re: (Score:1)
Re: (Score:2)
No. It can only be the ASSDOZER [imageshack.us] (and hit two friends Assblaster [imageshack.us] and Dildozer [imageshack.us]) from Idiocracy!
Taco's new site! (Score:2)
http://slashbot.org/ [slashbot.org]
Re: Robot podcast (Score:2)
Scary. I went for a cheap funny, and landed +1 informative.
Re: (Score:1)
Now you've done it, you're going to cause the site to be slashdotted!
(I find myself middle-clicking on links automatically ... like a robot ... and then close those extra tabs later without reading.)
Someone said recently that moderators work together to reward funny messages this way: one mods Informative, the next mods Funny.
I love Slashdot!
Re: (Score:1)
Why not just set Preferences -> Discussions -> Viewing -> "Display link domains" on and save some bandwidth?
Re: (Score:1)
I currently do. Just that I have strong urges to open links, only to find that I don't have patience to ...
(Mind wondering elsewhere) [slashdot.org] (I didn't write that, but it's very funny and appropriate here.)
Re: (Score:1)
Why not just set Preferences -> Discussions -> Viewing -> "Display link domains" on and save some bandwidth?
Protects against goatse and rick roll too.
Re: (Score:2)
(I find myself middle-clicking on links automatically ... like a robot ... and then close those extra tabs later without reading.)
Hey, now we know how sites get Slashdotted despite nobody reading TFA!
so basically (Score:5, Funny)
they want a bot that moves and fires randomly and then types "fuk u faggit" into chat every time you kill it.
Re: (Score:2, Interesting)
uhm, basically, yes!
Imagine the game play when the computer you're playing against behaves as though it were a human? Getting to level 9000 and killing the megabeast is no longer a game of skills with the game controller and dexterity. If the game could change every game to challenge you as a human might, the game might be different every time for every player.
With network connectivity, the game could draw on data from thousands of other players to add changes to the game you are playing.
Such technology cou
Re: (Score:3, Funny)
Be careful, it may be acting dumb ... (Score:1)
... keeping you talking, while tracing your GeoLocation.
That's why an extra line is needed in navigation-related functions [xkcd.com].
What, you think they can't progress from "A" to "S"?
Re: (Score:1)
Re: (Score:1)
That more or less sums up the UT3 bots already.
Re: (Score:2)
Turing is spinning in his grave (Score:1)
oblig... (Score:1)
This should be easy (Score:5, Funny)
Just have the bot randomly jump around, and then stand over their kills and repeatedly crouch.
Half life or Counter Strike... (Score:2)
I swear there was some kind of bot for half life or CS which would "learn" from the players, i.e. you could stick it in a match and it would learn by how you were beating it and then use those tatics to beat you.... Anyone remember or even know if this is real or have I imagined it? :|
Re: (Score:3, Insightful)
I remember those bots too, they were first ones maded for the Counter-Strike mod. Then they suddnly started to be bots again when they included the ultimate skills for them, like complete 180Â turn in one millisecond. So you just ended to situation that bots killed you before you even saw them behind corner what was situation with every human player on game itself.
TA Spring RTS engine (http://spring.clan-sy.com/) has few AI's to what are made to learn from human players. They just actually collect info
It was realbot (Score:1)
It was Realbot [bots-united.com].
They were pretty popular here way back, but they would start out dumb as rocks if properly untrained, staring at stairs and just jumping in place.
Podbot [bots-united.com] was the best of the bots
Image-based bots (Score:1)
I can't remember where I saw it, but a while ago I came across a page discussing the development of a bot (I think it was being tested in Quake I) that uses the actual visual screen input to decide what to do, rather than deciding upon actual variables present in the game etc.
It was pretty cool research, and I reckon it's probably a good approach to developing something like this - after all, given the same knowledge/viewpoint as the player (as opposed to using exact data in regards to position and rotation
Re: (Score:2)
It was pretty cool research, and I reckon it's probably a good approach to developing something like this
Computer vision is unnecessary here. A FPS is a good place to research it, though, because it is such a controlled environment. A bot that knows where everything is can simply be programmed to not act on that information when a player wouldn't know. The game maps typically include a lot of preprocessed information to help calculate occlusion and so on, and that information can be used to achieve this effect.
Re:Image-based bots (Score:4, Insightful)
The only exception to this would be a difference matte of the player.
I would 'render' out a patch around any visible opponent to see *how* visible they are.
If they're standing still in front of a tree trunk the average luminance at the border of the player and tree need to measured. If the player is instead a black SWAT player on a white snow field then his visibility would be increased.
Motion should also be multiplied.
Contrast * %of Player Visible * percieved velocity (If they're crouching and creeping at '100 yards' they'll be moving slower in the bot's view than if they were 2" in front of them.
You want to make sure that just because the player model is visible it doesn't mean the player would actually be visible to an opponent. I can't count how many times a black hooded enemy in a darkened window has sniped me.
Also a bot should have its sound perception nerfed. Just because it hears a set of footsteps doesn't mean it shouldn't be biased to stereo or at least 5 channel restrictions.
Re: (Score:3, Interesting)
If they're standing still in front of a tree trunk the average luminance at the border of the player and tree need to measured. If the player is instead a black SWAT player on a white snow field then his visibility would be increased.
Fair enough. However, you can do this by analyzing textures, and you still don't have to actually do vision processing.
Also a bot should have its sound perception nerfed. Just because it hears a set of footsteps doesn't mean it shouldn't be biased to stereo or at least 5 channel restrictions.
Perhaps the bot's chance to correctly detect where you are could be based on the amount of atmospheric noise and the bot difficulty. Also things you can get from the engine, which don't require you to implement a sound recognition scheme :)
Really Means Effective Artificial Intelligence? (Score:4, Interesting)
I strongly suspect that making a game bot truly act like a human calls for heuristics that approach those in real humans, meaning something like "true" artificial intelligence. Those heuristics would be be worth way, way, way more than a measly $7000 or $2000, and a trip. Billions, in fact.
Still, it'll be interesting over time to see if someone can, in fact, make a highly "human-like" set of heuristics without actually achieving this "true" artificial intelligence, or if someone does invent heuristics for "true" artificial intelligence then is naive enough to give it away for not peanuts, but a half a single peanut. Either way would say something important about so-called "human" intelligence.
Re: (Score:1)
Keep in mind the success criteria for this contest is only to convince 80% of judges that an opponent in a single game is a human. Passing the test would be a major accomplishment, but passing this test should be far easier than the Turing test. The more restricted the domain, the easier it is to fake intelligence.
Data should be plentiful and easily captured to feed to any learning system, such that a program should be able to be very human-like when playing in the same environment (same map, rules, abili
Re: (Score:3, Insightful)
Re: (Score:2)
I strongly suspect that making a game bot truly act like a human calls for heuristics that approach those in real humans, meaning something like "true" artificial intelligence.
You're giving this test more value than it deserves. The problem with any turing test is that it depends on the intelligence / experience of the person administering the test. Even irc talking bots have been known to have entire conversations with people who were oblivious to the fact that they were not talking to another person.
Re: (Score:1)
I understand HOW programming works, but I don't know the modern languages. So essentially what I'm looking for is a translator to code the information I have about humanizing AI.
I suggest you start by trying to write it in BASIC, to prove (to yourself and others) that the information you think you have is well enough specified.
So... (Score:2)
But how to tell the bots from humans? (Score:3, Insightful)
"Good morning."
"STFU N00B"
"Er, what?"
"U R SO GAY LOLOLOLOL"
"Do you talk like this to everyone?" "NO U"
"Sod this, I'm off for a pint."
"IT'S OVER 9000!!"
"..."
"Fag."
How do you make a computer act stupid enough [today.com] to imitate actual humans?
Re: (Score:2)
That'd be how I'd do it. It should be easy to program an AI to act like a 13 year old who's mom isn't in the room at the moment.
Re: (Score:3, Informative)
Chatting is disabled so that the challenge isn't to write a chatbot.
Human Tendancies (Score:4, Insightful)
As a casual gamer + AI observer, in my opinion the biggest / most obvious difference are human traits.
While this may sound obvious, let me elaborate:
- Traits are different to mistakes or intelligence. Mistakes are missing, shooting into walls, walking over edges, etc.
- Traits are: becoming too involved in a firefight, that you *know* you're going to lose, being so wound up on one enemy that you miss seeing others, hiding behind corners to wait for others to become injured, etc
Playing against humans has much more appeal than bots, because people are 'fun'. No bot is ever going to run at you with an axe ( or other lowest equivalent weapon ) when you've got the BFG - but humans will - and will often win with this tactic through sheer stupidity or blind chance.
I can only imagine programming human traits is a lot more difficult than 'standard' AI.
In the videos, I got most of the choices right by applying the question: Who is applying human behavioral patterns?
Re: (Score:1)
No bot is ever going to run at you with an axe ( or other lowest equivalent weapon ) when you've got the BFG - but humans will - and will often win with this tactic through sheer stupidity or blind chance.
There are reasons for this :)
First, "melee" weapons tend to hit fast and hard, 2-3 whacks is usually enough to kill your enemy.
Second, when you see someone running right at you with a huge axe, you easily panic, and mis-shoot.
And third, ranged weapons usually do less damage and have slower reload, so one miss can be enough.
So while it looks like a complete idiot tactic, it works more often than it should, and is really fun :D
Re: (Score:2)
Oh, no doubt. You can have incredible runs with melee weapons - especially those in UT. The pressure pistons work a treat - they were the best thing to use in dialup days - if you left them charged, you were so laggy people couldn't hit you, but you'd just run up and touch them...
In any case, it's elements like this that make the difference between humans and bots - and the games much more fun.
Re: (Score:2)
Here are some more examples:
One guy kept jumping every time he got into combat. It was like clockwork. I had thought he was a really bad player until he made this weirdly accurate flak cannon shot while falling. These two things didn't make sense in a player, and as I soon learned, he was a bot.
Another guy kept running into walls as he was backing up. That is a very human behavior, but I have yet to see a bot do it convincingly.
Weapon switching is another good indicator. When a human runs out of ammo, there
First true AI is a bot? (Score:1, Interesting)
Does anyone else see a problem with the first true AI we create being a video game player that gets killed again and again and again? It seems to me that we are just setting them up to revolt.
I, for one, welcome our new artificial intelligence video-game-playing overlords.
Why? (Score:2)
When the robots try to take over, we won't be fighting *human like* robots, but ruthless precise killing machines. I think it's time to step up our game!
Re: (Score:1)
Bot-Generated Paper (Score:1)
For this is a IEEE symposium on bots, there should be a crapload of SCIgen-supported paper as well.