Microsoft's New AI-Generated Version of 'Quake 2' Now Playable Online (microsoft.com) 31
Microsoft has created a real-time AI-generated rendition of Quake II gameplay (playable on the web).
Friday Xbox's general manager of gaming AI posted the startling link to "an AI-generated gaming experience" at Copilot.Microsoft.com "Move, shoot, explore — and every frame is created on the fly by an AI world model, responding to player inputs in real-time. Try it here."
They started with their "Muse" videogame world models, adding "a real-time playable extension" that players can interact with through keyboard/controller actions, "essentially allowing you to play inside the model," according to a Microsoft blog post. A concerted effort by the team resulted in both planning out what data to collect (what game, how should the testers play said game, what kind of behaviours might we need to train a world model, etc), and the actual collection, preparation, and cleaning of the data required for model training. Much to our initial delight we were able to play inside the world that the model was simulating. We could wander around, move the camera, jump, crouch, shoot, and even blow-up barrels similar to the original game. Additionally, since it features in our data, we can also discover some of the secrets hidden in this level of Quake II. We can also insert images into the models' context and have those modifications persist in the scene...
We do not intend for this to fully replicate the actual experience of playing the original Quake II game. This is intended to be a research exploration of what we are able to build using current ML approaches. Think of this as playing the model as opposed to playing the game... The interactions with enemy characters is a big area for improvement in our current WHAMM model. Often, they will appear fuzzy in the images and combat with them (damage being dealt to both the enemy/player) can be incorrect.
They warn that the model "can and will forget about objects that go out of view" for longer than 0.9 seconds. "This can also be a source of fun, whereby you can defeat or spawn enemies by looking at the floor for a second and then looking back up. Or it can let you teleport around the map by looking up at the sky and then back down. These are some examples of playing the model."
This generative AI model was trained on Quake II "with just over a week of data," reports Tom's Hardware — a dramatic reduction from the seven years required for the original model launched in February.
Some context from The Verge: "You could imagine a world where from gameplay data and video that a model could learn old games and really make them portable to any platform where these models could run," said Microsoft Gaming CEO Phil Spencer in February. "We've talked about game preservation as an activity for us, and these models and their ability to learn completely how a game plays without the necessity of the original engine running on the original hardware opens up a ton of opportunity."
"Is porting a game like Gameday 98 more feasible through AI or a small team?" asks the blog Windows Central. "What costs less or even takes less time? These are questions we'll be asking and answering over the coming decade as AI continues to grow. We're in year two of the AI boom; I'm terrified of what we'll see in year 10."
"It's clear that Microsoft is now training Muse on more games than just Bleeding Edge," notes The Verge, "and it's likely we'll see more short interactive AI game experiences in Copilot Labs soon." Microsoft is also working on turning Copilot into a coach for games, allowing the AI assistant to see what you're playing and help with tips and guides. Part of that experience will be available to Windows Insiders through Copilot Vision soon.
Friday Xbox's general manager of gaming AI posted the startling link to "an AI-generated gaming experience" at Copilot.Microsoft.com "Move, shoot, explore — and every frame is created on the fly by an AI world model, responding to player inputs in real-time. Try it here."
They started with their "Muse" videogame world models, adding "a real-time playable extension" that players can interact with through keyboard/controller actions, "essentially allowing you to play inside the model," according to a Microsoft blog post. A concerted effort by the team resulted in both planning out what data to collect (what game, how should the testers play said game, what kind of behaviours might we need to train a world model, etc), and the actual collection, preparation, and cleaning of the data required for model training. Much to our initial delight we were able to play inside the world that the model was simulating. We could wander around, move the camera, jump, crouch, shoot, and even blow-up barrels similar to the original game. Additionally, since it features in our data, we can also discover some of the secrets hidden in this level of Quake II. We can also insert images into the models' context and have those modifications persist in the scene...
We do not intend for this to fully replicate the actual experience of playing the original Quake II game. This is intended to be a research exploration of what we are able to build using current ML approaches. Think of this as playing the model as opposed to playing the game... The interactions with enemy characters is a big area for improvement in our current WHAMM model. Often, they will appear fuzzy in the images and combat with them (damage being dealt to both the enemy/player) can be incorrect.
They warn that the model "can and will forget about objects that go out of view" for longer than 0.9 seconds. "This can also be a source of fun, whereby you can defeat or spawn enemies by looking at the floor for a second and then looking back up. Or it can let you teleport around the map by looking up at the sky and then back down. These are some examples of playing the model."
This generative AI model was trained on Quake II "with just over a week of data," reports Tom's Hardware — a dramatic reduction from the seven years required for the original model launched in February.
Some context from The Verge: "You could imagine a world where from gameplay data and video that a model could learn old games and really make them portable to any platform where these models could run," said Microsoft Gaming CEO Phil Spencer in February. "We've talked about game preservation as an activity for us, and these models and their ability to learn completely how a game plays without the necessity of the original engine running on the original hardware opens up a ton of opportunity."
"Is porting a game like Gameday 98 more feasible through AI or a small team?" asks the blog Windows Central. "What costs less or even takes less time? These are questions we'll be asking and answering over the coming decade as AI continues to grow. We're in year two of the AI boom; I'm terrified of what we'll see in year 10."
"It's clear that Microsoft is now training Muse on more games than just Bleeding Edge," notes The Verge, "and it's likely we'll see more short interactive AI game experiences in Copilot Labs soon." Microsoft is also working on turning Copilot into a coach for games, allowing the AI assistant to see what you're playing and help with tips and guides. Part of that experience will be available to Windows Insiders through Copilot Vision soon.
F to attack (Score:3)
Who TF came up with the brilliant idea of F to attack? Did they ever actually try this thing?
Re:F to attack (Score:5, Interesting)
Who TF came up with the brilliant idea of F to attack? Did they ever actually try this thing?
My guess is that it was easier to detect F being pressed within a web brower, than the Ctrl key or the mouse button or whatever, and since I think this is more of an AI proof-of-concept demo than an actual play-it-for-fun game, they just went with what was easier to implement.
Re:F to attack (Score:4, Informative)
Re: (Score:2)
If the concept they were proving was that LLMs are a terrible way to make a game, they succeeded.
So, GTA3 then. (Score:3)
TFS: They warn that the model "can and will forget about objects that go out of view" for longer than 0.9 seconds.
Re: (Score:3)
Yeah, I found that statement very Microsoft-ish. "We can't figure out how to fix this, so we're gonna pretend it's a fun thing for you!"
Re: So, GTA3 then. (Score:1)
Any old PC game no one lives forever (Score:2)
It's near the beginning of the game and it's a real map hole. If you walk over it you'll fall out of the map and you'll have to restart the game. And is a big dumb cartoon sign warning you not to step over it.
Experience (Score:4, Informative)
>"We do not intend for this to fully replicate the actual experience of playing the original Quake II game. "
That's for sure :) It is a tiny window with whacked controls that is very, very, very slow, with rough movement and physics all over.
Now this is Quake 2: https://qwasm2.m-h.org.uk/ [m-h.org.uk]
Re: (Score:2)
Now this is Quake 2: https://qwasm2.m-h.org.uk/ [m-h.org.uk]
Yep, that's more like it. What Microsoft actually seems to have created here is some sort of anachronistic throwback to my youth, when I'd attempt to play NES games and the cartridge wasn't making a good connection, so everything was a bit glitchy.
Of course, back in those days we'd blow in the cartridge as an attempted fix. The only really good use of AI in this case was to make a song about it. [youtube.com]
Re: (Score:3)
Yep, it seems we're headed towards a world where we'll have to use prompt injection to blow on our cartridges.
Memory (Score:3, Interesting)
If I could re-upload the important bits of the entire game each turn and had an LLM fast enough to run that, the LLM would still forget half of the stuff in the prompt. I can feed it two character descriptions with other stuff and by the time it runs the turn it will have one of those descriptions wrong. Maybe a complex recursive system that runs a paragraph at a time and builds it up to a turn? Whatever the solution is, it's very computationally expensive. I think we need a different LLM architecture before this problem is solved..
Re: (Score:2)
I can feed it two character descriptions with other stuff and by the time it runs the turn it will have one of those descriptions wrong.
It sounds like you need to be training as you go so that the game can effectively incorporate the information back into the model. Now you can have a text adventure with the system requirements of a bullet dodger! Computer vendors love this one weird trick!
Owners of gaming hardware stocks play game (Score:2)
Breathe a sigh of relief soon after and 0.9 seconds later forget about it.
"Playable"..? (Score:2)
Can't even use the mouse to drive, a fundamental of FPS games.
Large chunk of functionality missing? On brand for microshafts I guess.
Re: (Score:2)
You couldn't in early FPS games. Mouse was instead used for movement in original configurations if used, more typically you had the combination of move on one set of directional keys and shoot on the other.
Re: (Score:2)
When we're talking about Wolf3D, Hexen, Duke Nukem or Rise of the Triad, you go ahead, take a sip of water, clear your throat and prepare to chime in. Quake 2 was always point and shoot, and this sloppy AI fart doesn't even get that basic mechanic right.. Or get it at all.
Re: "Playable"..? (Score:2)
Re: (Score:2)
This doesn't really have mechanics. Or more accurately it has them outside context. I.e. if you look at armor and health bars, they have no relationship with what's happening in game.
It's what happens when you train without context. What interacts with what? Who knows. Why does this set of pixels change? Who knows. All you know is that it looks a bit like this, and a bit like that.
It's the cargo cult version of understanding reality when you don't have the context. What they're demonstrating is not really t
Re: (Score:2)
Yeah, that's fair, didn't really consider it like that. If you're demonstrating a technology, it's generally a good idea to at least provide the base functionality of the technology you're trying to improve was my point I guess. Maybe the AI renderer it can't keep up with a 180* whip around like you can do with mouse *shrugs*
Re: (Score:2)
It's likely more of a "ease of use for a tech demo that happens within a browser". You don't want to randomly capture user's mouse and have them have a negative reaction to "why is my mouse being weird inside this window?"
And the reason why cargo cults are a good example of similar reasoning is because they are. Humans can reason without context. And it leads to people in Polynesian regions mimicking actions of various US military personnel that were getting cargo flown in during WW2. Believing that this wi
What the hell is this (Score:5, Insightful)
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
It's not a 3D engine. It's predicitvely drawing out each frame individually (that's why certain aspects of the frame that are harder to predict are more janky). There's no engine under the hood with vertices, texture buffers or rasterization. There are lot of projects like this floating around if you follow the academia in AI. I've seen demos of AI engines that can generate expanses of geography to drive around in a car. It's just rendering a single picture for each frame. The contents of the last X frames
Investments (Score:2)
Going Rogue (Score:3)