Slashdot

http://tasvideos.org/ [tasvideos.org]

TAS = Tool Assisted Speed Runs. Basically you program controller inputs (at very slow speeds) and then play them back at 1:1 speeds and you watch a pre-programmed controller run through an entire game as quickly as possible. There are runs for basically most of the more popular NES and SNES games as well as other games. Pretty interesting stuff and usually a daunting task is creating a TAS of a game.

500 dollars for the winner, but you are expecting evolutionary neural nets, genetic programming, fuzzy logic, and temporal difference learning.

The temporal difference between the effort to build such an AI and 500 bucks seems a little too great.

Yet you only post this on slashdot with 13 days before the deadline. You couldn't have posted it here back in May? (the earliest date a post seems to have in your google group).

I highly like this competitions idea, but I won't participate, because the deadline is far, far too soon.

I mean, I am supposed to understand their framework and implement, test and tweak an artificial intelligence for a pretty complicated task like this in a month (let alone, 2 weeks), with my rusty java, rusty AI-knowledge (I'd try emergent behaviour, probably)? Sorry, but this is just plain impossible, since there is enough work to do from the university and other hobby projects. Give me until, like. Christmas and I'd try.

Plus, the time shortens even further, as it appears that there are documentation issues, so one would probably have to work out how the game state is given to the AI.

So overall: very interesting, but too short for someone who actually has other work to do

http://2009.rl-competition.org [rl-competition.org]

We also used Infinite Mario Bros, but combined it with the RL-glue coding framework to make the interface easier. That way, a well-coded agent is automatically compatible with any other domain that is RL-glue compatible.

The prizes were also comparable: ~$450 for the first place team, ~$250 for the second place team.

The results were interesting: far from developing interesting and novel RL algorithms, most competitors used clever feature engineering combined with dimensionality reduction to reduce the full Mario problem to a simpler one that could be solved efficiently using existing RL algorithms that are robust and well understood.

One of the big lessons that we took away from this was that we haven't solved the mechanism design problem of competitions in AI. While Mario sounds like a good "grand challenge" problem for RL / AI, it turns out that simple heuristics work pretty well. I think this is a common problem for most of these competitions -- there's the Trading Agent Competition, there's Netflix, there's the General Game Playing Competition, etc. They all have the same goals, and they all have the same problem: competitors engineer algorithms to solve the competition, not to spur progress in general AI. These games are all a proxy for what we really care about (like the Turing test), and the proxy isn't perfect (like the Turing test).

I think the only way to get around this is to craft a domain that mimics the real world, because then if anyone "solves the competition," you've made progress on what you really care about.

It would be interesting to design a competition with these goals in mind. Maybe an extraordinary complex simulator based on a physics engine (Bullet or Havok) would be a step in the right direction -- different objects with continuous, high-dimensional state spaces and complex material properties (some are soft, some are rigid, some break, etc); interesting physical interactions between objects (collisions, joints, hinges, stacking, breaking, etc.); multiple levels of spatio-temporal abstraction (from low-level motor control to abstract tasks) and a strong vision component. Now that would be a cool competition!

David Wingate
wingated@mit.edu