Forgot your password?
typodupeerror
Networking Games Technology

Predictive Modeling To Increase Responsivity of Streamed Games 120

Posted by Unknown Lamer
from the so-predictable dept.
jones_supa (887896) writes Streaming game services always bump up against a hard latency limit based on the total round-trip time it takes to send user input to a remote server and receive a frame of game data from that server. To alleviate the situation, Microsoft Research has been developing a system called DeLorean (whitepaper) using predictive modeling to improve the experienced responsiveness of a game. By analyzing previous inputs in a Markov chain, DeLorean tries to predict the most likely choices for the user's next input and then generates speculative frames that fit those inputs and sends them back to the user. The caveat is that sending those extra predictive frames and information does add a bandwidth overhead of anywhere from 1.5 to 4 times that of a normal streaming game client. During testing the benefits were apparent, though. Even when the actual round-trip time between input and server response was 256 ms, double-blind testers reported both the gameplay responsiveness and graphical quality of the DeLorean system were comparable to a locally played version of the game.
This discussion has been archived. No new comments can be posted.

Predictive Modeling To Increase Responsivity of Streamed Games

Comments Filter:
  • by He Who Has No Name (768306) on Tuesday August 26, 2014 @12:01AM (#47753967)

    Nobody can predict when I will suddenly chase a mammoth with a fork while buck naked.

    Nobody.

    • It can't predict all three at once, I'll grant. But if you've already given the commands to strip naked and wield a pitchfork, then of course it can speculatively run the choice for pushing the joystick toward the mammoth.
      • If you've already pushed your joystick towards the mammoth AND are known to frequent slashdot, one could speculate that you will be crazy enough to strip down and wield a pitchfork.

        Wait, you said fork. Hmmm..that's a tough one.

    • When did they add pitchforks to Skyrim? Or is this in Elder Scrolls Online?

    • by gl4ss (559668)

      they pretty much only need to guess what happens if you pressed lmb.

      of course, if they were testing with something with only digital on/off inputs, the whole thing becomes much more easier than sending 100 different frames for 100 different possible mouse moves(and calculating those frames at the server end too! it doesn't only need more bandwidth it needs the game that is being streamed do a lot more.. and have it's engine rewritten too... which makes this a lot less appealing).

      it's the analog inputs that

      • by tepples (727027)
        I think they're moving the mouse a "typical" amount and rounding the predicted move command to either no motion or the predicted amount of motion. Besides, in first person games, screen motion due to mouse panning is easy to predict, as John Carmack pointed out.
    • by 91degrees (207121)
      Of course they can - you do it all the damn time!

    • Assuming you can do just about anything in a game, yeah nobody can predict that.

      On the other hand, in a game, with an "open world" design that actually has a corridor structure, yes they can predict a lot of movement, choices etc.

      The system may backfire if you intentionally do things unrelated to game advancement but that's a different story.
      • Quakeworld had limited prediction as an "improvement" over original Quake. It had a tough time with rocket jumping. More often than not the screen would freeze and a few seconds later, you'd find yourself running in a corner, the jump never having occurred, or missed badly.

    • you are chasing a wooly mammoth buck-naked with a fork. we determine you need titanium toe rings with 6-carat blue diamonds. why? hell, just because. your credit card on file has been decremented ten million dollars. now your avatar jingles as you run.

      perhaps you want an elephant gun now? please enter a new valid credit card number...

  • by Anonymous Coward

    So...branch prediction. Yea, that does seem like an interesting idea. Take it to the next step and you can render n-frames ahead based on all probable inputs and try your best to continuously pair down future paths as you get actual user input. But realistically that'd increase the bandwidth requirements 1000% or more which in most circumstances will make the situation worse. Not to mention the difficulty of actually writing games/environments to deal with these speculative paths or the generally choppi

    • Precisely. It's branch prediction, and they've effectively already taken it to the "next step" you outlined, since that's precisely what they're doing, with the frames from the predicted paths being sent to you ahead of schedule so that the moment you provide input they can display the frames that correspond to that input, rather than having to wait for the round trip to the server. One interesting technique they noted is that they can send some additional data along with the frames, allowing your local PC

    • by Darinbob (1142669)

      I suspect from the description that these are not your normal open 3D games, but perhaps games with the dreaded evil quick time events (QTE). Otherwise there are just to many variables to base a prediction on. 3 axes of freedom, an inventory full of weapons and tools, 10 dialogue choices... You could have thousands of branches to choose from.

      My guess is that the game predicts you'll buy the DLC to enable the quest that the NPC is offering you, and so is predictively entering in your credit card number (

  • EA better not make games streaming only as there simcity must be on line only not only was a lie the game also sucked big time.

  • Does this mean that Microsoft will assume responsibility for creating unresponsible persons?

  • by Nyder (754090) on Tuesday August 26, 2014 @12:59AM (#47754159) Journal

    The biggest problem for streaming games is going to be the ISP.

    They don't like it when people use netflix, think about something like a game, where you are sending more stuff, not to mention upstream is being used more on these.

    Streaming games would kill any download limits you have on your ISP and pretty much all of them have some sort of limit in place.

    But what is the good of this if your ISP cuts your account, or nutters the connection so you can't play because you went over the 200-400gb limit for the month?

    • by tepples (727027)
      I guess you'd stream a game for about three days to see if you like it, after which you'd buy your own copy. But wouldn't buying your own copy run up a 30 GB bandwidth bill by itself, now that console games ship on BD-ROM and PC games are BD-ROM-sized downloads?
      • even so you can download an game off peak when some ISP are cap free and you can say download an game / parts of it at places with free WiFi.

        • even so you can download an game off peak when some ISP are cap free

          If you plan to go this route, satellite is in my experience far more likely to include unmetered off-peak use than cellular.

          and you can say download an game / parts of it at places with free WiFi.

          If your computer happens not to be a laptop, which is likely for a gamer because laptop GPUs tend to be underpowered in both senses, watch people point and laugh at someone bringing in a desktop computer to download a game. That's the vibe I get from Not Always Right [notalwaysright.com], Geekologie [geekologie.com], and Paradoxoff [paradoxoff.com].

          • you can download on a laptop and move the files over to your desktop with gameing hardware. Just saying even if disks go away there are ways to get download only media with out needed a good link at the place you want to view it / play it at. Even with steaming movies / tv / etc (other then live) having a buffer is good. But something like this kills that idea and makes so that you need good link 100% of the time you want to play the game. Also hope that you are the only person in the home playing the game

    • think about something like a game, where you are sending more stuff, not to mention upstream is being used more on these.

      Its not "that much", let me explain:
      - 4x keyboard inputs = 4x char (4 bytes)
      - 2x mouse inputs X/Y = 2x float (8 bytes)
      - Thats only a total of 12 bytes for client inputs that needs to be sent.

      The issue is the update rate or "tick rate".
      Ideally you need to match the update rate to the framerate being received for smooth input response. In standard practice, its wise to update the input loop outside of the code loop. On standard games installed on a system, this can actually provide more input updates than ac

      • by hibiki_r (649814)

        The bandwidth problem is not on the way out (it's bigger than you think, but it's still small), but on the way in. It's a 1080p video stream that has to be compressed on the fly that cannot do any significant amount of buffering. Netflix already eats bandwidth for lunch, and that with compression algorithms that can run for as long as you want to optimize bandwidth use. So we get weaker compression, and we send a user 4 frames for every frame they see, so 8 times the bandwidth of Netflix for the same image

    • by LienRag (1787684)

      Streaming games would kill any download limits you have on your ISP and pretty much all of them have some sort of limit in place.

      Actually... no.
      Data caps on wired internet mostly exists in "capitalist" america, where the market is so free that it doesn't mind belonging to an oligopoly.
      The rest of the world has competition, and usually no data caps (except on mobile services usually, which sucks too).

  • Such creativeness with the language.

  • by Loki_666 (824073) on Tuesday August 26, 2014 @01:30AM (#47754253)

    Its even easier to pull complete support for the game when it depends on their servers!!! This way they can sell Super Mega Game X+1 to those who used to play Super Mega Game X, forcing them to pay for the new shiny versions, which is really almost identical, with no compelling new features, except maybe slightly better graphics and a +1 to the title.

    I now stay 100% away from any games which demand an internet connection to play. If a game is online only, but supports local server or can be emulated via Hamachi or something, then i'm cool with it. Otherwise no.

    If this means i can't play game X, then i won't die from the lack of it. Plenty of other games out there which I can play, and play how i want.

    • I would think one benefit would be that you can allow for a subscription service where people could try lots of different games or play portions of games without having to download hundreds of gigs of data for games they only play a few times or when they are only playing a specific mission and don't need all the maps or game modes available.

      I'm not saying this is what everyone wants or that it works for all use cases, but I imagine this is one scenario.

      • by hibiki_r (649814)

        Let's take a humongous game: 10 gigs. How long does it take for an in game, low compression, high quality video stream to be a whole lot more downloading time than those 10 gigs? And remember that predictive streaming can take 4 times more bandwidth than regular streaming. I would be surprised if you don't get to 10 gigs in a few hours.

        I just can't see a world where, on average, you save bandwidth by streaming games. Quite the opposite, in fact: This system is a non-starter in a world of bandwidth caps.

  • Hang on a minute:

    "During testing the benefits were apparent, though. Even when the actual round-trip time between input and server response was 256 ms, double-blind testers reported both the gameplay responsiveness and graphical quality of the DeLorean system were comparable to a locally played version of the game."

    Gameplay responsiveness, sure, but... graphical quality? If I was testing a system like this, I'd be asking about that as a way to identify people who were just agreeing it was better because they thought it was what I wanted to hear. Graphical quality should not be affected by how quickly the streamed game can respond. There's something fishy about this.

    • Compression quality can increase with longer processing time. If they are pre-processing frames then they can spend a few extra milliseconds on compression encoding and decoding quality.

    • by 91degrees (207121)
      It is double blind so we can control for people being nice.

      Graphical quality is subjective, so people are going to be influenced by whether they enjoyed it more. Chances are they'd also report better sound effects, and even a more enjoyable journey home.
  • by Animats (122034) on Tuesday August 26, 2014 @02:48AM (#47754445) Homepage

    Back in the 1990s, Microsoft developed something similar. Their idea was to render frames in layers, with the more distant or less active layers rendered less often. if the viewpoint changed, the background layers were scrolled, rotated, or transformed to match, rather than being re-rendered immediately. It never caught on, because graphics hardware became fast enough to re-render everything on for each frame.

    This new thing is similar. Mispredicted frames are viewpoint-warped as a temporary measure so the user sees something. The image is wrong, but close enough to look OK until a new rendered frame is sent. It looks OK for Doom, on which it was tested, because Doom is mostly about the shooter and the opponents moving; there's not much general activity in the background. GTA IV/V would probably look much worse than normal.

    The whole concept represents a desperate attempt to make something "cloud-based" that shouldn't be.

    • graphics hardware is so cheap these days. Even your $85 Amazon Fire TV can render some pretty awesome games...and it just keeps improving...so it seems like the real benefit isn't graphics per se but in AI and CPU processing, in maintaining massive worlds, and in enabling a uniform gaming experience across platforms/clients.

    • by drinkypoo (153816)

      It never caught on, because

      ...they were trusting Cirrus Logics to put together two out of four chips, and they failed to execute in a timely fashion. By the time they had a prototype, the hardware was outdated.

  • bad name (Score:3, Funny)

    by Charliemopps (1157495) on Tuesday August 26, 2014 @09:08AM (#47755789)

    Apparently the authors of this research have never actually driven a DeLorean. It was stainless steel, weight 3000lbs, and only has 130hp.

    So basically it's a car the weight of a Buick with the engine of a Geo Metro.

    • by neminem (561346)

      And apparently you don't know anything about pop culture, or else are being willfully obtuse? It's pretty clear they didn't name it after the car, but after a particular famous *use* of the car to travel through time. Doesn't matter how crappy your car is, if it's also a time machine.

    • by mcrbids (148650)

      FYI: The larger Geo Metro 1.3 liter engine produced 70 HP. Cars in the 3,000 lb range fit in the "mid sized sedan" range which typically have 150-225 horsepower.

      Yes, it was under powered, but it was not a "Geo Metro".

  • This has been around for a while now [wikipedia.org], the only difference here seems that is it's being applied to streaming games and done on the server side for the player, instead of the player side.
  • Soon enough these streaming game services will be sending entire binaries to run on the client in order to render something quicker. First it will be the background binary, then pre-send all the textures. Soon enough the entire game will be running on the client.

Money will say more in one moment than the most eloquent lover can in years.

Working...