Does Mathematical Tuning Make Games Better? 109
simoniker writes "What do game designers need to know about statistics? Age Of Empires DS designer Tyler Sigman focuses on statistical topics that he believes should be understood by game designers, in a new article. His reasoning: 'In the game I just finished, we recorded data from play sessions and then set challenge levels in the game based upon the mean and standard deviation values from those recorded data. We set Medium difficulty to be equal to the mean values, Easy difficulty to be equal to the mean minus a certain amount of standard deviations, and then Hard difficulty equal to the mean plus a certain amount of standard deviations.' Would all games be better if they were tuned mathematically?"
Leave out "Mathematical" (Score:5, Insightful)
Since tuning is all about improving the feel of the game to the humans who will interface with it, it all depends upon the creator for how he wishes to accomplish this. In this case, the creator was looking for sweet spots that he was able to find through mathematical manipulation of sampled data. In other cases, using math to tune the results might give the game a clinical feel; something that's generally bad for video games. (Unless you're playing Trauma Center.
So the question is pretty much moot. Creating a good game is an art form, but even art can benefit from a few structural calculations.
Re: (Score:2, Insightful)
More games need that kind of accountability.
Re: (Score:2)
Mathematical tuning is a good place to start, but basing everything off of it is NOT a good idea. You always need a base point, and what better base poitn to use then some cold numbers. However from there you need to test it out and see if the game is still FUN, and if it is not, what are the problems?
I actualy have seen a few instances where developers knew the game was unbalanced (win/loss records always favored one team), however left it that way becasue the game was actualy more fun to
Re:Leave out "Mathematical" (Score:4, Insightful)
Case in point: I once had a cribbage game where you could play against the computer and set different levels of difficulty. I quickly discovered that "Expert" level just meant that the computer got better hands more often -- it had nothing to do with the quality of the computer's strategy. After getting lousy hands several games in a row while the computer consistently drew hands like 4-5-5-6, I simply stopped playing. While "Expert" level was certainly harder, it was also not fun to play.
So, while TFA has a point about statistics being important for game design, that's not much more profound than saying that vision is important for driving cars well.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
IIRC, it wasn't a name brand like Hoyle, though. I think it came in one of those cheap game packs (25 popular card games for $10!). You get what you pay for, I guess.
Re: (Score:1)
Make with the proof or I shall forever assume that you were whipped 2 or 3 times in a row by the computer and then came up with this "the computer gets a better hand theory" in desperation.
Re: (Score:2)
Now I am not an expert cribbage player by any means. I do remember, though, that I was beating the computer consistently on the Intermediate level, which was also not very fun and why I chose to bump it up to the Expert level.
It's also possible that the hands would have evened out had I played 100 games on the expert level. But after the first dozen games or so, it see
Re: (Score:1)
So I set the Mac to easy, and let the Windows dominate until it was far beyond hopeless, something like 3 chips left for the Mac side with the Windows side very bloated. Then I jammed the Mac one up to expert, and it actu
Re: (Score:2)
For example, in the first Warcraft, the
Re: (Score:2)
Re: (Score:2)
A background in Econometrics or statistics would be a huge boon to MMO developers in charge of game balance. Massaging numbers to identify leading factors and weighting their impact on a result is the core function of this area of study. It's perfectly suited for MMO balancing.
A heavily dumbed down example:
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
'Day of Defeat' seems to have found a nice balance, people pick their guns based somewhat on preference, but also upon the needs of the map. Some maps you want every sniper you can get, others they're virtually useless.
right idea, wrong direction (Score:1)
designers can balance a map out by varying the level design, so that one weapon can be the advantage in one area, while other areas are better suited by other weapons... giving one weapon the advantage at all times it not an enjoyabl
Re: (Score:2)
So there is a pay off for a rather heffty reward.
On the flipside I can easily balance a rocket launcher vs a MG, how? For the RL tweak up reload times, tweak down projectile speed and walking speed,
Re: (Score:2)
Machine guns (light machine guns at that, 7.62 or more mm caliber) have wind adjustments on sights, can shoot cover fire at 600m, have iron sights for up to 1000m, and can fire rather precisely (antisquad automatic fire) at 300m. Machine guns (7.62mm - not squad automatic weapons) can fire to 1500m.
SMGs and pistols are only usable when you see the white of the enemies' eye
Re: (Score:2)
That can ruin the replay value, but replay value is somewhat inimicable to the "Aha!" moment you get when you realize the key to a map. Sure, after you figure out the killer strategy on a map, it isn't challenging anymore, but you get a rush from figur
Re: (Score:2)
Re: (Score:2)
Bunny-Hopping is a VERY specific thing, related to the HL engine, having to do with gaining speed due to the funky way airspeed works.
Peopel bouncing around like idiots is NOT bunny hopping, it is people bouncing around like idiots (and thus making them EASY targets that can't acuratly fire back).
Re: (Score:1)
It was in Quake 1 first. Since Half-Life is a hybrid of Q1 and Q2, I'm not surprised it exists in HL.
Re: (Score:2)
No, it's not - and it's been around a lot longer than 'Half Life'.
Sadly, In most games bunny hopping doesn't make it much harder for people to fire back, which is the problem (e.g. as in the BF series - though Dice did eventually add measures to BF2 to this effect to try and discourage it).
One reason it's particularly annoying is it can screw up hit detection (in large multip
Re: (Score:2)
To be honest, I don
Re: (Score:1)
Everything revolves around what is being mathematically tuned. Is the intelligence of an object being tuned (cool) , or is the object just being given a x% production/whatever boost (sucks). Games that use the latter approach have that clinical feel to them, but this is the easiest method of tuning.
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
I would agree with the notion that tuning matters a lot in PvP, to make sure there isn't one "best unit" or "best weapon" that can be consistently used to win against peo
Re: (Score:3, Interesting)
BlackEmperor writes: "Is the intelligence of an object being tuned (cool)" or is the object just being given a x% production/whatever boost (sucks).
"Intelligence" generally does not have a "knob" that you can simply tune up and down. How smart a character acts is an emergent behavior that depends on many other factors in the system, themselves which include many tunable knobs like "x% whatever boost", and complex dynamic behavior scripted into the code.
That said, you can still add more layers to tune
Re: (Score:2)
"Tuning" in general might be a good thing, but you need to base you optimization on something - why not math? The
Re: (Score:2)
That's not really "math" in the way you mean it. It's a form of computations, yes, but closer to an analog computer than a digital calculator. The brain does arithmetic quite poorly.
Because the results of some random calculation will feel "cold" and not at all enjoyable. Or to put it another way,
statistical tuning (Score:1)
really it's just tuning through statistical means.
tuning == always important
tuning through statistical means == might work, assuming that statistical means yields something fun for the player.
Re: (Score:1)
Mathematical or statistical? (Score:1)
developers (Score:3, Insightful)
Re: (Score:2)
It's not trial and error. It's a binary search algorithm [wikipedia.org] that executed within O(log n) time. :P
Think of it like turning a knob back and forth, getting closer to the setting you feel is best. The "best" setting is the one with the most appeal to humans, and may not be the most realistic. Unless you're programming an accurate simulation, that is. In which case the players are usually willing to put up wit
Re:developers (Score:4, Insightful)
Think of it like turning a knob back and forth, getting closer to the setting you feel is best.
That method will only deliver a local maximum of a polynomial function. If your game has any complexity at all, your proposed method is even less useful than trial and error.
Re: (Score:2)
That method will only deliver a local maximum of a polynomial function. If your game has any complexity at all, your proposed method is even less useful than trial and error.
I think you've hit the nail on the head here. Mathematical tuning can help fine-tune game parameters for one player one strategy or another. These are like finding a local minimum. But clever players will come up with clever strategies that would represent other isolated, remote minima.
Even so, there are mathematical methods to search for these other minima as well, and I suppose they could be applied to the topic of game balancing. I very much doubt anyone has tried this yet (with any success at least
Re: (Score:1)
Re: (Score:2)
You are right about everything else
You call that "mathematical modelling"?! (Score:4, Interesting)
Wow. If this is "mathematical modelling", then me swapping the coffee mugs out for wine glasses in my kitchen cubbard would be "advanced sphere packing analysis and optimization".
Game tuning as more art than science. The goal is not to create an interestingly distrubuted difficulty curve, but to create an "easy", "medium", and "hard" amount of enjoyable challenge. Huge amounts of time can be (and frequently are) wasted focusing too-strongly on a "cool" and intriguing difficulty model that some under-experienced junior designer is all fired up about, instead of keeping the focus tightly and solely on the how the game actually feels.
Re: (Score:1)
I found the title to be an overstatement as well. The designers just used simple statistics to adjust the difficulty and the article is describing it like they needed a PhD in math to figure it out.
That being said, it's still a great idea. It may seem obvious after the fact but this is probably something a lot of game companies
Re: (Score:3, Interesting)
I am constantly amazed at how much game programmers know about the mathematics and algorithms for
Re: (Score:2)
I am constantly amazed at how much game programmers know about the mathematics and algorithms for computer graphics, and how little they know of everything else.
As a graphics programmer myself (though not for games), I can say that it's really mostly geometry. In order to get speed, the geometry is tortured into a form that can be difficult to understand, but ultimately it's just geometry.
Even relatively simple mathematical concepts like sampling theory are above the heads of most day-to-day graphics
Re: (Score:1, Insightful)
I am constantly amazed at how much game programmers know about the mathematics and algorithms for computer graphics, and how little they know of everything else.
Ah, I see that you are from the CS department at CMU. As it seems you are unaware let me relate to you a fundamental flaw in the general social environment of game development: game developers are not computing scientists. For the most part those constituting 'old hat' developers are self-taught and have developed a chauvinistic narrow-minded vie
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
I bet Easy isn't actually easy. (Score:5, Interesting)
Unfortunately I suck at games. My coordination is all over the place. I have NO patience. I play games for a laugh, I don't want to invest a great deal of time learning a game or practising it. I want to pick it up, play for a while, and be entertained. As a rule I always play games on Easy because I don't want a challenge. I don't want to get frustrated playing the same level over and over. I want that feeling of progression like I'm getting somewhere. I can honestly say that if I get stuck for more than an hour in a game it gets turned off and never switched on again. I make a mental note not to buy a game from the same people again.
Easy is for people like me. Lazy, good-for-nothing "casual" players who have no skill to speak of and a life of some sort that means there isn't the time to learn perfection. I expect Easy to be easy. I very much doubt that "mean minus standard deviation" of some enthuiastic professional testers or Beta players is really going to be down at my level.
Please, for the love of Mario, when you're writing a game, sit your mother down in front of it for a few hours and tweak the difficulty of "Easy" to something she can cope with. That way I might buy your sequel.
Alternatively, give me God mode.
Re: (Score:2)
Easy should be just that - easy. Makes me feel like I'm having fun, not getting a whipping.
-WS
Re: (Score:2)
Re: (Score:2)
You mean like in Crash Bandicoot where you get a free Aku Aku to protect yourself if you die too many times before reaching the next checkpoint?
Yes, that is a nice feature. Of course, I wouldn't have needed it if I had figured out earlier tha
Re: (Score:2)
I really liked the game, despite the less-than-glowing reviews - it's everything I enjoy in a Rare title, in exactly the same way I'm loving Kameo and Viva Pinata now. But that insane difficulty spike stopped me from seeing half of more of the game.
Re: (Score:1)
Re: (Score:2)
I caught a *lot* of crap about having to play DMC in easy mode, but what can I say? I suck
-WS
Re: (Score:1)
Feeling like you're getting somewhere and playing a level over and over again are not mutually exclusive, try playing e.g. Ninja Five-O for the Gameboy Advance, you die often but each time you get a bit further and then finally get to the end with a feeling of great accomplishment. I don't like frustrating r
Re: (Score:3, Interesting)
Re: (Score:1)
Re: (Score:1)
Nonetheless, as the kind of guy who DOES like to do so, I find your statements amazingly offensive.
Of course, I am at least self aware enough to not think ill of you and your lifestyle. Different strokes for different folks.
I have no idea why I even made this post...
Re: (Score:2)
And unfortunately, it's people like yourself that take away what makes games great: Interactivity and the depth of control and manipulation you have over you virtual character. That sense of control, the challenges that you overcome while learning a game are what make games great to begin with.
Imagine prince of persia on "automatic" where the computer navigates the level and fights for you. Not a game I would want
Re: (Score:1, Insightful)
In YOUR opinion.
Some people find micromanagement tedious as hell, some people get off on having to tell each individual unit in a strategy game what to do. Some people have physical disabilities that mean they CAN'T,
Re: (Score:2)
Angband has a fully automated AI player. While it works, it does have some quirks and sometimes gets perma-killed.
That being said, fully automation only works if the gameplay style has a predictable combat system that the player is fully able to control. As soon as there's something that wrecks automation, the player needs to switch to damage control mode.
Here's some examples of some nasty things
Ever play an RTS? (Score:2)
You could play Dune 2, in which you must give every single unit an order. So: Click the troop, click where you want him to go. Rinse and repeat for however many you have. I think they at least attack automatically...
Or, you could play Starcraft. Click+drag to select a squad -- up to 12 units, where some units (dropships, overlords) can carry other units inside them. Click where you want them to go, watch them attack anything they find (or run, depending on what mode you have them set to).
Or, you could pla
Re: (Score:2)
The truth is gaming is about interacitivity, and the fact that interacitivity is going the wa
Re: (Score:2)
I'm sorry, what? I mean, yes, it's required for it to be fun, and the levels are designed around you having those features. But so what?
And so did Dune, as I said in my post. So, it is actually possible to control every unit individually. It just m
Re: (Score:2)
I think you should rethink the meaning of the word "game". Chess players don't want to have to play against Deep Blue every time; it's more fun to be able to play a game where you can win, and you shouldn't *have* to invest time into enjoyment. Varying difficulty settings are there for those who want a greater challenge. For you the challenges that you overcome while learning a game are what makes games great to begin with. For others, games are a way to relax away from t
Re: (Score:1)
No chess game ups the difficulty by adding pawns and bishops to the other side (more opponents but rules otherwise stay the same and equal), or giving the knights the ability to jump in L or F or Z patterns, or lets the rooks move on diagonals like queens, (different, "easier" rules for computer) or, worst of all, gimping your side (harder rules for you = doubly lame because not only do you not play by the same rules, but y
Re: (Score:2)
/dies
Re: (Score:1)
I think gaming is about having fun. You like to put a lot of effort in it, to win, to beat the computer. For others it is laughing at that little funny guy that jumps in the air when you push the spacebar.
Re: (Score:2)
Interesting that you mention Prince of Persia, since that game (SoT, but also the former 2D ones) is in large part what I call an "on-rails jump'n run" and its good exactly because of that. What I mean with 'on rails' becomes obvious when you compare it with a standard FPS.
In a FPS you basically control a
Re: (Score:2)
I agree but more and more games are taking players OUT of the game. See FF12, where you just run around navigating the whole game. That was the most tedious FF I have ever played, I was not involved all I did was navigate I did not feel my actions had impact on the world, as everything was automatically done for me.
The problem with automating gaming too much is t
Re: (Score:2)
I expect Easy to be easy. I very much doubt that "mean minus standard deviation" of some enthuiastic professional testers or Beta players is really going to be down at my level.
Maybe, but that would be a problem of not getting a representative sample. Statisticians aren't normally fools. They know that anyone willing to sign up for a beta isn't going to be a good sample of the people who buy the game. They can either try to somehow correct for this based on previous data (beta players are 30% faster at fi
A lifetime gamer who sucks? (Score:2)
Alright, you do have some valid points.
One thing I should mention, some people do improve their games by making them easier -- or at least consistent in their difficulty. Jak & Daxter was mostly easy, partly because you could skip most of the game, by choosing easier alternatives -- it's a pretty open game; each area needs x number of Power Cells, and there are probably 2x or 3x quests you can do in that area which give you a Power Cell. But some parts were hard; for instance, the final boss is very di
Statistics to Tune (Score:2)
While the article doesn't present it well, I think that the author probably is very good at tuning games. He doesn't come right out and
Re: (Score:1)
Re: (Score:2)
Another interesting thing to note is that extensive use o
Too easy? (Score:2)
Not exactly rocket science (Score:1)
Sims Designer Chris Trottier on Tuned Emergence (Score:3, Informative)
Sims Designer Chris Trottier on Tuned Emergence and Design by Accretion [donhopkins.com]
The Armchair Empire interviewed Chris Trottier [armchairempire.com], one of the designers of The Sims and The Sims Online. She touches on some important ideas, including "Tuned Emergence" and "Design by Accretion".
Chris' honest analysis of how and why "the gameplay didn't come together until the months before the ship" is right on the mark, and that's the secret to the success of games like The Sims and SimCity.
The essential element that was missing until the last minute was tuning: The approach to game design that Maxis brought to the table is called "Tuned Emergence" and "Design by Accretion". Before it was tuned, The Sims wasn't missing any structure or content, but it just wasn't balanced yet. But it's OK, because that's how it's supposed to work!
In justifying their approach to The Sims, Maxis had to explain to EA that SimCity 2000 was not fun until 6 weeks before it shipped. But EA was not comfortable with that approach, which went against every rule in their play book. It required Will Wright's tremendous stamina to convince EA not to cancel The Sims, because according to EA's formula, it would never work.
If a game isn't tuned, it's a drag, and you can't stand to play it for an hour. The Sims and SimCity were "designed by accretion": incrementally assembled together out of "a mass of separate components", like a planet forming out of a cloud of dust orbiting around star. They had to reach critical mass first, before they could even start down the road towards "Tuned Emergence", like life finally taking hold on the planet surface. Even then, they weren't fun until they were carefully tuned just before they shipped, like the renaissance of civilization suddenly developing science and technology. Before it was properly tuned, The Sims was called "the toilet game", for the obvious reason that there wasn't much else to do!
Here are some questions and answers from the interview with The Sims designer Chris Trottier:
[...]
Q: On paper, a game where you simulate daily life doesn't sound that interesting. Yet The Sims is really fun to play, so much so that it is now the biggest-selling PC game ever. Although any development team working with Will Wright has to feel confident in the product they are creating, has the unbelievable popularity of the franchise shocked even the development team?
A: Absolutely. When I was first assigned to The Sims, it was not-very-affectionately-known within the company as "the toilet game." Will Wright had tremendous stamina for the risk involved with trying something very new, but there were certainly a lot of head-scratchers both on the team and outside of it. In all honesty, the gameplay didn't start to really come together until a couple of months before ship. Being involved in that tuning process, and seeing the game take shape from what had previously been a mass of separate components, was one of the most powerful experiences of my career.
[...]
Q: What makes The Sims massively popular with female gamers, who traditionally don't make up a big number of gameplayers?
A: It's so hard to answer that question without making broad, sweeping statements that anyone of my gender would probably resent. But... I can say that there are several untraditional forms of gameplay in The Sims. For instance, there are many people who spend most of their time decorating and redecorating their homes. Since there's so much user-created content being posted on websites, they spend a lot of time collecting more looks to add to the game. There are also a lot of people who enjoy having a fantasy life where they get to call the shots... for good or for bad. I've heard a lot of stories
Re: (Score:2)
Long story short: you throw all the game elements in a pot, then figure out how to fit them together in a way that's "fun". Failure to do this results in a failure to
Re: (Score:3, Insightful)
to be honest, I read the name and the premise and decided not to download. there is no reason for the premise to be exciting. it's more of a demonstration when you throw all the elements for one meal into one pot, and all the elements for another meal into another pot, and th
Re: (Score:1)
Bad to do automatically (Score:2)
Basically, the game "watched" me get better at playing a certain level of the game... unsuccessfully. But, apparently it saw that I was doing soooo well, that it decided to increase the difficulty without telling me. Which actually made me continue to fail to complete the level. Quite frustrating, not to mention annoying to have to keep an eye on the difficulty level so that it doesn't go beyond what you want.
IMO, Ratchet Deadlocked did
Re: (Score:1)
AOE DS was tuned? (Score:1, Offtopic)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
How much tuning can you do these days? (Score:1)
Difficulty levels (Score:1)
my take (Score:2, Insightful)
Dystopia (Score:1)
If mod developers relying on volunteers can use statistical analysis, its pretty muc
Tuning Wii Sports (Score:2)
For example in Tennis it's become noticably harder at the 500 level, while at the 0 level the computer is trivial to beat.
I think satisfaction happens when the game is hard enough that failure is a realistic possibility, but you still tend to win more often. You can tune on the players performance, or a sample audience performance, it doesn't really matter.
Some hardcore games tune ultimate hardness with the intent of
Already Online: Trueskill / etpub / Guild Wars (Score:1)
More complex modeling than means and standard deviations have gone into improving online games (unless the author was simplifying things by saying means and standard deviations).
For example both Trueskill [microsoft.com] and etpub [etpub.org] use a Bayesian form of Arpad Elo's rating system to rank and match players.
I did some work modeling kills and wins in Enemy Territory that yielded interesting insights about map- and weapon- balance in that game.
At arena.net, there is at least one employee whose sole job is to model the ass
Stupid (Score:1, Troll)
Re: (Score:2)
Jagged Alliance 2.x (Score:2)
How would you ever tune this game? I have been playing it for nearly five years, including all of its mods and new developments. Tuning it is nearly impossible because the tactics employed by new players are so vastly different from veterans that I cannot fathom how it could be done mathematically.
Nevertheless, if you coders want to go at it, its open source. Go to the Bear's Pit.
how else are they tuned ? (Score:2)
Statistics should only add polish (Score:1)