Predictive Modeling To Increase Responsivity of Streamed Games 120
jones_supa (887896) writes Streaming game services always bump up against a hard latency limit based on the total round-trip time it takes to send user input to a remote server and receive a frame of game data from that server. To alleviate the situation, Microsoft Research has been developing a system called DeLorean (whitepaper) using predictive modeling to improve the experienced responsiveness of a game. By analyzing previous inputs in a Markov chain, DeLorean tries to predict the most likely choices for the user's next input and then generates speculative frames that fit those inputs and sends them back to the user. The caveat is that sending those extra predictive frames and information does add a bandwidth overhead of anywhere from 1.5 to 4 times that of a normal streaming game client. During testing the benefits were apparent, though. Even when the actual round-trip time between input and server response was 256 ms, double-blind testers reported both the gameplay responsiveness and graphical quality of the DeLorean system were comparable to a locally played version of the game.
There is no way this could work for me when I play (Score:5, Insightful)
Nobody can predict when I will suddenly chase a mammoth with a fork while buck naked.
Nobody.
Re:There is no way this could work for me when I p (Score:5, Insightful)
Or vice versa.... (Score:2)
If you've already pushed your joystick towards the mammoth AND are known to frequent slashdot, one could speculate that you will be crazy enough to strip down and wield a pitchfork.
Wait, you said fork. Hmmm..that's a tough one.
Re: (Score:2)
When did they add pitchforks to Skyrim? Or is this in Elder Scrolls Online?
Re: (Score:2)
I'm sure there's a pitchfork mod. If not now, within the next ten minutes at least.
Re: (Score:3)
they pretty much only need to guess what happens if you pressed lmb.
of course, if they were testing with something with only digital on/off inputs, the whole thing becomes much more easier than sending 100 different frames for 100 different possible mouse moves(and calculating those frames at the server end too! it doesn't only need more bandwidth it needs the game that is being streamed do a lot more.. and have it's engine rewritten too... which makes this a lot less appealing).
it's the analog inputs that
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Assuming you can do just about anything in a game, yeah nobody can predict that.
On the other hand, in a game, with an "open world" design that actually has a corridor structure, yes they can predict a lot of movement, choices etc.
The system may backfire if you intentionally do things unrelated to game advancement but that's a different story.
Re: (Score:1)
Quakeworld had limited prediction as an "improvement" over original Quake. It had a tough time with rocket jumping. More often than not the screen would freeze and a few seconds later, you'd find yourself running in a corner, the jump never having occurred, or missed badly.
especially like the wallet integration part (Score:2)
you are chasing a wooly mammoth buck-naked with a fork. we determine you need titanium toe rings with 6-carat blue diamonds. why? hell, just because. your credit card on file has been decremented ten million dollars. now your avatar jingles as you run.
perhaps you want an elephant gun now? please enter a new valid credit card number...
Re:Why? (Score:5, Interesting)
Why on earth would you want to do this? Run the damned thing locally
Let me count the guesses: A publisher paranoid about prohibited copying may be willing to license its game at a lower price if the game program never leaves the server. Or it might be cheaper and faster to send a video stream than to send sufficiently powerful hardware and 50 GB of game at once. Or sufficiently powerful mobile hardware might not even exist. Or it might want to ensure that all players connected to the same server have comparable lag and the same inability to install cheat mods.
Re: (Score:3)
Substitutability (Score:2)
The customer WILL buy our car because otherwise, he won't be able to drive.
Video games are less substitutable [wikipedia.org] than passenger cars. Someone who wants Super Smash Bros. 4 isn't going to be satisfied with Destiny nor vice versa.
Re: (Score:2)
We just had the discussion yesterday how game makers are more and more copycats who latch on whatever area of video gaming seems to promise a sale. It might not work for certain franchises, but do I really care just which zombie shooter I'm playing?
Re: (Score:2)
That's because up until a couple of years ago, game budgets were getting bigger and bigger. They hit the same problem as hollywood: When your next release is going to cost many millions of dollars to make, you can't risk that kind of money on something new and untested. You have to go for something with a history of market success, like a sequel or a franchise installment. That's not so much of a limit in games now because of the rise of mobile games and electronic distribution (Thank you, Steam), both of w
Re: (Score:2)
and they stop playing after they see the bandwidth bill also what about people who's only HSI is satellite?
People with download caps as low as 10GB? even 50GB is low for something light this. The lag on satellite is to high for something like this as well.
Satellite is a rounding error (Score:5, Insightful)
Australia is a rounding error (Score:2)
250ms ping to a US/EU server is about average since very few of the major titles have servers in Oz, SE Asia or Hawaii is normally is good as it gets from the arse end of the Earth. You know you have a really clean connection if it's under 200ms.
The lag from Oz to anywhere else on the planet is largely a true physical limit imposed by the speed of light, no amount of bandwidth will fix it. Anything that "tricks the users brain" int
Re: (Score:2)
A video game publisher is likely to view people stuck on satellite as a rounding error.
ROFL. Apple became one of the most highly valued companies on the planet dealing with so-called rounding errors. How many people use OS X? How many people used iOS devices (at first) and are they dominant even now?
Sure, coast along happy in ignoring rounding errors, but those small things WILL kill you sooner or later. Kind of like Ebola. ;)
Re: Why? (Score:2)
There are still people with bandwidth limits?
Are you sure you don't live in the past?
Re: Why? (Score:2)
Download limit rather.
Download limits are very much still a thing here (Score:2)
There are still people with [download] limits?
Yes. Comcast still has the 300 GB per month limit in many markets, and cellular has a cap two orders of magnitude smaller than even that.
Are you sure you don't live in the past?
For someone born in a country whose home Internet pricing expectation is stuck in the past, such as the United States, Canada, or especially Australia or New Zealand, it can be expensive and a pile of red tape to relocate to a country in the present.
Re: (Score:2)
Limiting the market (Score:2)
Re: (Score:2)
Then I guess I'll have to deal with it the way I deal with "always online" DRM:
I'll have to abstain.
Re: (Score:2)
A publisher paranoid about prohibited copying may be willing to license its game at a lower price if the game program never leaves the server.
*stifles laughter*
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Imagine if the Ubisoft always-on DRM were an inherent, unremoveable aspect of the game system rather than just something tacked on to a few individual games after the fact, such that Ubisoft couldn't even begrudgingly neuter it in a patch. Well, a streamed game is even worse than that would be.
All you get is streaming video/audio and all the lag you'd expect (including controller lag), which
Re: (Score:2)
Publishers want to rent you a game, not sell you one. Renting is lower up-front cost so it gets more people looking at the game (no one makes demos anymore), and those that get hooked on the game spend much more than the normal $50 price.
I think these are all single player games, so lag vs fairness wouldn't matter so much.
Re: (Score:2)
Let me count the guesses: A publisher paranoid about prohibited copying may be willing to license its game at a lower price...
LOL no. They will just see the increased profits as profits that were not lost to piracy.
What they fail to understand is this will not bring increased revenue or increased profits... but that is for another discussion.
Re: (Score:2)
Because it's easier to extract money from people continuously if you can deny them their crack^Wgame at will.
Re: (Score:1)
Why on earth would you want to do this? Run the damned thing locally and be done with it. If it aint broke...
Ummm......online gaming?
Or do you honestly expect Microsoft, Sony et al to put their servers in your livingroom?
Please RTFA or at least RTFS before you hit the Post button....
Re: (Score:2)
Regular online games don't have this problem as frames are rendered locally. Only streamed games will benefit benefit, and that's what the question was referring to. His question isn't addressed in the article, so your call to read it is completely irrelevant.
Re: (Score:2)
Re: (Score:2)
But this is specifically about streamed games. Locally rendered games (as with WebGL) can render the scene immediately in response to the player's input, so they don't have this problem. The lag is still there, but you only have to deal with the physics side of it, rather than dealing with the physics *and* the audio and video.
The scenario you describe doesn't make any sense because the idea is that you choose the frame (or scene) based on the inputs on the local machine. So if you're rendering it anyway th
Re: (Score:2)
Re: (Score:2)
The predictive algorithms for the scene would be deterministic, so you may as well perform them on the local machine. I still don't see how you propose this would work, so maybe I should ask this question specifically: What would be the criteria upon which the local client would select one of the pre-constructed scenes?
Re: (Score:2)
Why on earth would you want to do this?
For example to stream windows only games from my basement server to lightweight linux-based HTPC in my living room.
Re: (Score:2)
Selling a million game consoles will produce less benefit to fewer users than taking the equivalent hardware and putting it into a server farm.
A console in the average game console owner's home sits unpowered for the vast majority of the day. That hardware in a server farm will be used 100% of the day, meaning more hardware will be available for more people.
A server farm can be far more energy efficient than a million consoles.
Obviously there are tradeoffs, but to dismiss the concept outright is not helpfu
Branch Prediction (Score:1)
So...branch prediction. Yea, that does seem like an interesting idea. Take it to the next step and you can render n-frames ahead based on all probable inputs and try your best to continuously pair down future paths as you get actual user input. But realistically that'd increase the bandwidth requirements 1000% or more which in most circumstances will make the situation worse. Not to mention the difficulty of actually writing games/environments to deal with these speculative paths or the generally choppi
Re: (Score:2)
Precisely. It's branch prediction, and they've effectively already taken it to the "next step" you outlined, since that's precisely what they're doing, with the frames from the predicted paths being sent to you ahead of schedule so that the moment you provide input they can display the frames that correspond to that input, rather than having to wait for the round trip to the server. One interesting technique they noted is that they can send some additional data along with the frames, allowing your local PC
Re: (Score:1)
I suspect from the description that these are not your normal open 3D games, but perhaps games with the dreaded evil quick time events (QTE). Otherwise there are just to many variables to base a prediction on. 3 axes of freedom, an inventory full of weapons and tools, 10 dialogue choices... You could have thousands of branches to choose from.
My guess is that the game predicts you'll buy the DLC to enable the quest that the NPC is offering you, and so is predictively entering in your credit card number (
EA better not make games streaming only as there (Score:2)
EA better not make games streaming only as there simcity must be on line only not only was a lie the game also sucked big time.
Microsoft's responsibility (Score:1)
Does this mean that Microsoft will assume responsibility for creating unresponsible persons?
Re: (Score:3)
Similar in many regards, yes. Quakeworld, from what I understand and recall, focused on two things: predicting where you were going so that it could prepare those parts of the world in advance, and predicting where others were going so that it could draw them as accurately as possible on your screen even if the connection was slow (I may be incorrect about Quakeworld having this feature, where a predictive bot run locally on your machine effectively replaces the other players for a few ms at a time, but I k
Re: (Score:2)
You have problems with consistent client behavior over 140 ping. This research document supposedly went up to either 200 or 400, depending on what RTT means.
Here is the real problem: What happens if you introduce random packet drop to the system? Over mobile broadband, shitty long line connections, Australias landline, and more, you get packet loss. A small amount with a random amount added.
Does this research paper even touch on the subject?
Re: Awesome! (Score:2)
Pff you dont get it.
Microsoft will be going all the way to 200 MS with this amazing new technology!
Your move Apple and Google.
ISP don't like the streaming (Score:3)
The biggest problem for streaming games is going to be the ISP.
They don't like it when people use netflix, think about something like a game, where you are sending more stuff, not to mention upstream is being used more on these.
Streaming games would kill any download limits you have on your ISP and pretty much all of them have some sort of limit in place.
But what is the good of this if your ISP cuts your account, or nutters the connection so you can't play because you went over the 200-400gb limit for the month?
Re: (Score:2)
Re: (Score:2)
even so you can download an game off peak when some ISP are cap free and you can say download an game / parts of it at places with free WiFi.
Shame in free Wi-Fi for a desktop? (Score:2)
even so you can download an game off peak when some ISP are cap free
If you plan to go this route, satellite is in my experience far more likely to include unmetered off-peak use than cellular.
and you can say download an game / parts of it at places with free WiFi.
If your computer happens not to be a laptop, which is likely for a gamer because laptop GPUs tend to be underpowered in both senses, watch people point and laugh at someone bringing in a desktop computer to download a game. That's the vibe I get from Not Always Right [notalwaysright.com], Geekologie [geekologie.com], and Paradoxoff [paradoxoff.com].
Re: (Score:2)
you can download on a laptop and move the files over to your desktop with gameing hardware. Just saying even if disks go away there are ways to get download only media with out needed a good link at the place you want to view it / play it at. Even with steaming movies / tv / etc (other then live) having a buffer is good. But something like this kills that idea and makes so that you need good link 100% of the time you want to play the game. Also hope that you are the only person in the home playing the game
Re: (Score:2)
think about something like a game, where you are sending more stuff, not to mention upstream is being used more on these.
Its not "that much", let me explain:
- 4x keyboard inputs = 4x char (4 bytes)
- 2x mouse inputs X/Y = 2x float (8 bytes)
- Thats only a total of 12 bytes for client inputs that needs to be sent.
The issue is the update rate or "tick rate".
Ideally you need to match the update rate to the framerate being received for smooth input response. In standard practice, its wise to update the input loop outside of the code loop. On standard games installed on a system, this can actually provide more input updates than ac
Re: (Score:3)
The bandwidth problem is not on the way out (it's bigger than you think, but it's still small), but on the way in. It's a 1080p video stream that has to be compressed on the fly that cannot do any significant amount of buffering. Netflix already eats bandwidth for lunch, and that with compression algorithms that can run for as long as you want to optimize bandwidth use. So we get weaker compression, and we send a user 4 frames for every frame they see, so 8 times the bandwidth of Netflix for the same image
Re: (Score:1)
Care to redo your calculations with 90-240 (60 fps * [1.5 to 4]) frames of 1080p video/second?
I'am assuming you want the calculations for the video bandwidth at 60fps for 1080p?:
1920x1080 = 2,073,600 pixels
3x 4byte float for RGB channels per pixel = 24,883,200 bytes, per frame
60 frames a second = 1,492,992,000 bytes/ second
With prediction (*4 maxium value), upto = 5,971,968,000 bytes/second
No doubt they will be compressing that data. Probably adding extra latency to group frames into the compression ;)
Re: (Score:1)
Streaming games would kill any download limits you have on your ISP and pretty much all of them have some sort of limit in place.
Actually... no.
Data caps on wired internet mostly exists in "capitalist" america, where the market is so free that it doesn't mind belonging to an oligopoly.
The rest of the world has competition, and usually no data caps (except on mobile services usually, which sucks too).
"Responsivity?" Really? (Score:2)
Such creativeness with the language.
Re: (Score:2)
Yeah [merriam-webster.com]. And yes, language is meant to be used creatively.
Additional "benefit" (Score:4, Insightful)
Its even easier to pull complete support for the game when it depends on their servers!!! This way they can sell Super Mega Game X+1 to those who used to play Super Mega Game X, forcing them to pay for the new shiny versions, which is really almost identical, with no compelling new features, except maybe slightly better graphics and a +1 to the title.
I now stay 100% away from any games which demand an internet connection to play. If a game is online only, but supports local server or can be emulated via Hamachi or something, then i'm cool with it. Otherwise no.
If this means i can't play game X, then i won't die from the lack of it. Plenty of other games out there which I can play, and play how i want.
Re: (Score:2)
Re: (Score:2)
I dunno how DLC works on consoles, but in the PC world, DLC is actually installed to your computer and is generally playable whether or not the developers are continuing to run some kind of server for the game.
That's how it works on consoles too. The guy you're replying to is ill-informed.
allows for on demand gaming (Score:2)
I would think one benefit would be that you can allow for a subscription service where people could try lots of different games or play portions of games without having to download hundreds of gigs of data for games they only play a few times or when they are only playing a specific mission and don't need all the maps or game modes available.
I'm not saying this is what everyone wants or that it works for all use cases, but I imagine this is one scenario.
Re: (Score:2)
Let's take a humongous game: 10 gigs. How long does it take for an in game, low compression, high quality video stream to be a whole lot more downloading time than those 10 gigs? And remember that predictive streaming can take 4 times more bandwidth than regular streaming. I would be surprised if you don't get to 10 gigs in a few hours.
I just can't see a world where, on average, you save bandwidth by streaming games. Quite the opposite, in fact: This system is a non-starter in a world of bandwidth caps.
Improved graphical quality? (Score:1)
Hang on a minute:
"During testing the benefits were apparent, though. Even when the actual round-trip time between input and server response was 256 ms, double-blind testers reported both the gameplay responsiveness and graphical quality of the DeLorean system were comparable to a locally played version of the game."
Gameplay responsiveness, sure, but... graphical quality? If I was testing a system like this, I'd be asking about that as a way to identify people who were just agreeing it was better because they thought it was what I wanted to hear. Graphical quality should not be affected by how quickly the streamed game can respond. There's something fishy about this.
Re: (Score:2)
Compression quality can increase with longer processing time. If they are pre-processing frames then they can spend a few extra milliseconds on compression encoding and decoding quality.
Re: (Score:1)
Graphical quality is subjective, so people are going to be influenced by whether they enjoyed it more. Chances are they'd also report better sound effects, and even a more enjoyable journey home.
Microsoft did something like this once before (Score:5, Interesting)
Back in the 1990s, Microsoft developed something similar. Their idea was to render frames in layers, with the more distant or less active layers rendered less often. if the viewpoint changed, the background layers were scrolled, rotated, or transformed to match, rather than being re-rendered immediately. It never caught on, because graphics hardware became fast enough to re-render everything on for each frame.
This new thing is similar. Mispredicted frames are viewpoint-warped as a temporary measure so the user sees something. The image is wrong, but close enough to look OK until a new rendered frame is sent. It looks OK for Doom, on which it was tested, because Doom is mostly about the shooter and the opponents moving; there's not much general activity in the background. GTA IV/V would probably look much worse than normal.
The whole concept represents a desperate attempt to make something "cloud-based" that shouldn't be.
cheap graphics hardware (Score:2)
graphics hardware is so cheap these days. Even your $85 Amazon Fire TV can render some pretty awesome games...and it just keeps improving...so it seems like the real benefit isn't graphics per se but in AI and CPU processing, in maintaining massive worlds, and in enabling a uniform gaming experience across platforms/clients.
Re: (Score:2)
It never caught on, because
...they were trusting Cirrus Logics to put together two out of four chips, and they failed to execute in a timely fashion. By the time they had a prototype, the hardware was outdated.
Re: (Score:2)
TV is. Music generally is. Why are games so special?
TV and Music is a non-responsive, non-interactive recording.
It's not like it is a physics issue, just cheap-ass network operators not laying lines from this CENTURY, hell, technically even last, some seriously still use aluminium.
From a physics perspective the age and material of the lines are irrelevant. Signals travel at nearly the same speed. The latency is due to the physical limitations of the hardware and routing infrastructure. This is mitigated by placing the server closer to the client, which obviously costs the game provider much more.
Or just ads at loading screens, again, as long as it was instant (which it would be since it is based their servers)?
Actually, loading screens would be one thing that cloud computing might make economical to seriously reduce.
bad name (Score:3, Funny)
Apparently the authors of this research have never actually driven a DeLorean. It was stainless steel, weight 3000lbs, and only has 130hp.
So basically it's a car the weight of a Buick with the engine of a Geo Metro.
Re: (Score:3)
And apparently you don't know anything about pop culture, or else are being willfully obtuse? It's pretty clear they didn't name it after the car, but after a particular famous *use* of the car to travel through time. Doesn't matter how crappy your car is, if it's also a time machine.
Re: (Score:2)
FYI: The larger Geo Metro 1.3 liter engine produced 70 HP. Cars in the 3,000 lb range fit in the "mid sized sedan" range which typically have 150-225 horsepower.
Yes, it was under powered, but it was not a "Geo Metro".
Quake 3 & Duke Nukem 3D (Score:2)
Re: (Score:2)
Duke Nukem 3D..... client-side prediction [wikipedia.org].
[citation needed]
Soon enough (Score:2)
Soon enough these streaming game services will be sending entire binaries to run on the client in order to render something quicker. First it will be the background binary, then pre-send all the textures. Soon enough the entire game will be running on the client.
Re: (Score:1)
Girls, girls. You're both idiots.
Re: (Score:3, Funny)
"I have no clue what I'm reading about but am angry about it none the less!"
- Random internet commenter
Re: (Score:2)
Now they want to make games essentially play themselves.
No, they don't. Read what this is about.
What happens when the player produces input the software does not expect?
If it matches none of the speculated actions then the speculative frames are discarded or it can use what they call a "misprediction compensation technique" to build the correct frame from the speculated frames. How about reading what this is about and then asking specific questions about it?
Few gamers want to play games any more, they just want instant gratification trophies. There's more focus in achievement trophies in modern games (especially AAA games) than there is on actual gameplay.
What is that based on? For years we had the promise that one day the level of immersion and the quality of the output would be such that it would be able to produce a sort of "int
Re: (Score:2)
I thought Bioshock was bad when they made a game that you literally could not lose at. It was impossible to die or fail.
It worked out quite well for Ron Gilbert and Monkey Island in 1990.
The pleasures to be found in playing a game like The Dig or Grim Fandango lie in exploring the worlds their authors create.
If you begin with something as richly imagined as Rapture or the airship city Columbia the mistake is trying to shoehorn the game into the narrow confines of a first person shooter,
Re: (Score:2)
Bioshock games are story-driven. You fight to bring a sense of immersion and interaction, but a fight on which the player can be stuck for hours would become a barrier to the story rather than a way to draw them in.