5958481
story
Comments: 188 +- Games: Blizzard Offers Look Inside WoW At GDC on Friday September 18, @12:01PM
background: url(//a.fsdn.com/sd/topics/topicgames.gif); width:68px; height:77px;
games
background: url(//a.fsdn.com/sd/topics/topicit.gif); width:75px; height:61px;
it
background: url(//a.fsdn.com/sd/topics/topictech2.gif); width:60px; height:80px;
technology
Yesterday morning at GDC Austin, Blizzard's J. Allen Brack and Frank Pearce took to the stage to finally give a peek inside the inner workings of World of Warcraft. Tipping the scales at around 4,600 people utilizing 20,000 computer systems and 1.3 petabytes of storage, Blizzard has created a raging behemoth. The Online Network services group alone has "data centers from Texas to Seoul, and monitor over 13,250 server blades, 75,000 cpu cores, and 112.5 terabytes of blade RAM. [Pearce] points out the picture of the GNOC (Global Network Operations Center) in their slideshow, a data core that even has televisions tuned to the weather stations. They use those to ensure that conditions of the data center are up to their standards; with only a staff of 68 people they ensure connectivity across the globe for the numerous WoW servers."
Related Stories
: by
No man in the world has more courage than the man who can stop after
eating one peanut.
-- Channing Pollock
All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2010 Geeknet, Inc.
Story at 11... (Score:5, Funny)
Massive online game requires massive ammount of servers, bandwidth and people to maintain.....
Re: (Score:2)
ummm...how would I mod this...
Re:Story at 11... (Score:4, Funny)
With wild abandonment?
Parent
Re:Story at 11... (Score:4, Funny)
+5 He-Man And The Masters Of The Obvious!!1
Parent
Re: (Score:3, Interesting)
Re: (Score:2, Funny)
Re:Story at 11... (Score:5, Funny)
All you need is a post-it note saying "do not turn off!! SERVER!!!!"
Parent
Re: (Score:3, Insightful)
Then you find the one person who thinks unplugging the power strip to plug it into another outlet will not shut off the computer
Re: (Score:2, Funny)
You do not need such massive infrastructure to run a MMO, its just being wasteful about resources. Sure, the uptime could suffer a little bit and its possible you would sometimes run over the allowed bandwidth with Comcast, but you CAN run these things just fine on your living room behind the TV. If you want redundancy, you could have another server at your friends place.
Just because its "cool" and you have some money, it doesn't mean you have to waste it. Hell, even Slashdot runs just fine on CmdrTaco's mo
Re: (Score:3, Insightful)
Re: (Score:2)
Re:Story at 11... (Score:5, Insightful)
...yes. The hard part of a Massively Multiplayer Online Game does in fact come from the Massively part.
Parent
Re: (Score:2)
And a follow-up story at 12 - This just in: Computer nerds find massive amounts of servers, bandwidth, and people maintaining systems to be interesting. Some of them like hearing the details in success stories of large scale infrastructure. Even more so when the company itself is prominent or interesting.
Tune in tomorrow for our shocking revelation that some nerds also feel the need to mock these stories as un
Re:Story at 11... (Score:4, Interesting)
The interesting question with the amount of people working at blizzard is that will indie mmo's stand any change?
World of Warcraft utilizes 20,000 computer systems, 1.3 petabytes of storage, and more than 4600 people.
That's quite hard to compete with, and it only seems to be growing. Even other big MMO's have trouble competing with WoW, with Eve Online pretty much the only true competitor (and its more targeted for hardcore players)
Parent
Re: (Score:2)
Can someone explain this to me? I thought there were millions of WoW users? 4600 seems miniscule to me, especially when the same sentence says that they have 20,000 computer systems to serve these 4600 people... I don't understand.
Re: (Score:2)
4600 is the amount of people working on WoW, including the support staff and "two full-time lore historians, keepers of blizzard's past.". I'd like to have a work title "Warcraft Lore Historian"
Re: (Score:2)
As far as the millions- it takes 1 normal middle-high end computer to maintain 500 people logged in. If they have 20,000 servers, they can support their entire population (10 million) logging in at once (assuming they're evenly spread between servers).
Re: (Score:2)
Yes, WoW coders are actually from GetAFreelancer.com.
Well, Look at Their Monthly Revenue (Score:4, Insightful)
Re: (Score:2)
Did you read the article? They employ around 4000+ people on WoW alone.
Support staff:
Brack went on to talk about the customer support staff, a group with 2,056 game masters, 340 billing managers, and a host of other background staffers. These tireless staffers also work from locations around the world, ensuring that any local variations in culture (or the game) are respected.
Re:Well, Look at Their Monthly Revenue (Score:5, Informative)
Parent
Re:Well, Look at Their Monthly Revenue (Score:5, Funny)
Parent
Re: (Score:2, Insightful)
That's the first time in a long time that a Yakov Smirnov joke has made me do something other than want to slap people.
You get my vote...
Re:Well, Look at Their Monthly Revenue (Score:5, Insightful)
"In Soviet Russia, the government controls the corporations."
is also pretty good, imho.
Parent
Re:Well, Look at Their Monthly Revenue (Score:5, Interesting)
Except ASIA is a large portion of the subscriber base (the 10 million number they like to quote a lot)and doesn't pay much per month at all. Blizzard licenses the game to ISPs and other partners that resell the game service as part of their offerings.
So that part of it IS known, and you should factor that into your equations. Monthly income off WoW is nowhere near $120 million.
Parent
Re: (Score:2)
Re: (Score:3, Informative)
No, the 11.5 million number is accurate as at least up to 23Dec2008:
From: http://www.blizzard.com/us/press/081121.html [blizzard.com]
World of Warcraft's Subscriber Definition
World of Warcraft subscribers include individuals who have paid a subscription fee or have an active prepaid card to play World of Warcraft, as well as those who have purchased the game and are within their free month of access. Internet Game Room players who have accessed the game over the last thirty days are also counted as subscribers. The above d
Should I Be Concerned... (Score:4, Interesting)
...that WoW servers are guarded and maintained better than DoD networks?
Re:Should I Be Concerned... (Score:5, Insightful)
Parent
Re: (Score:2)
Re:Should I Be Concerned... (Score:4, Informative)
Check out the terms of service on your bank account. You might be shocked to learn the bank isn't responsible for your financial losses. Often, they specifically exempt themselves from all responsibility relating to fraud, mistakes, and/or computer errors. If they cash a bad cheque, you are on the hook.
There is a reason why people that survived the Great Depression hide money under their mattresses.
Parent
All I read... (Score:5, Funny)
Re: (Score:2)
Re: (Score:2)
Re:All I read... (Score:5, Informative)
I was told by someone at blizzard that they essentially implemented a fix across all battlegroups (which for those who don't know is a collection of realms at one data center) so you shouldn't see the error anymore. The problem was that each realm had a set amount of blades (something like 14?) for instances. Lower population realms didn't use hardly any of that capacity - whereas high population realms there wasn't enough. Well any good server admin knows you never can tell if a low population realm becomes a high population realm or visa versa so clearly you can't build these realms based off that alone - the app needs to scale accordingly.
The fix was that now all instances belong to a pool of servers now - which will eventually allow instance sharing across realms (that is - a party of players on different realms) once its switched on.
Parent
Re: (Score:2)
His brain is probably still lagging a bit; possibly stuck in a mental Battle Ground.
"Only" 68 people? (Score:2, Insightful)
with only a staff of 68 people
How does it take 68 people to monitor that few servers, and most of them BLADES?!? The writers have apparently never worked in a large network environment (not that I'd expect that they would have, being writers and all). But seriously... that's not really that many servers for a large online service, it really shouldn't take that much work to keep it all running unless it's horribly designed.
Eh well, if they have the cash flow to retain that many warm bags of mostly water, m
Re: (Score:2)
If you were to assume that those 68 folks are at a central monitoring facility AND spread across multiple datacenters (Yes Timmy I said turn the power button off, pull out the bad blade, replace it with a good one) and you have 3 shifts (+1 Weekend) working 24/7 you might have 17 people on duty at any one time.
They also said responsible for connectivity, so a few of these guys are rebooting routers and load balancers, not just blades.
Re: (Score:2)
How does it take 68 people to monitor that few servers, and most of them BLADES?!?
The summary:
The Online Network services group alone has "data centers from Texas to Seoul, and monitor over 13,250 server blades, 75,000 cpu cores, and 112.5 terabytes of blade RAM. [Pearce] points out the picture of the GNOC (Global Network Operations Center) in their slideshow, a data core that even has televisions tuned to the weather stations. They use those to ensure that conditions of the data center are up to their standards; with only a staff of 68 people they ensure connectivity across the globe for the numerous WoW servers."
Is a bit confusing, to say the least. But it sounds like those 68 people are not monitoring a pile of blades in a single location. It sounds like those people are monitoring server scattered across the globe. Further, I doubt if all 68 of them work 24/7 - while the servers will need to be monitored nearly 24/7. So I'd assume there are various shifts through the day/week.
All things considered... 68 people doesn't sound like an absurd number to me.
Small programming dept (Score:3, Interesting)
The programming department currently consists of 32 people, and envelopes systems, tools, gameplay, server technologies, and UI.
I know adding more developers can slow down production in the short term, but 5 years on I would think they would have been able to scale their programming staff up a bit more by now. New ui elements (gear manager, quest helper, even voice chat) have tended to be late and light on features, so thats one area I would think could benifit from more bodies in the future.
Re: (Score:2)
I think you need to (re)read 'The Mythical man month', more people does not equal more productivity. Not to mention that some features have been 'late' not because they were difficult to code, but likely because they didn't feel they were necessary yet. Not to mention (2) that every new thing creates more QA, and that the game needs to remain accessible for new players, not only for people that have been there for years.
No... (Score:2)
It can hurt in the long term and the short term. You get too many bright people on a project and it takes forever to reach consensus on entirely too much. Particularly if each is passionate about the entire product beyond their small piece and have strong opinions/vision about how it should be done. Especially if they are users/fans of their own software.
Re: (Score:2)
MMOs have one two weaknesses:
The first is that every new feature needs to be tested for software regressions thoroughly. Not to mention unexpected code or content interactions that players can use to gain a disproportionate disadvantage.
For example, there was a bug with one game where if one knew the constants that skills were named, one could try manually type in the skill name and learn it, even if that skill was never a part of that class. So you would have warriors running around who could open a figh
Pictures? (Score:2)
So wheres the pictures?
Brack?? (Score:2)
Production Director Brack and Game Director Tom Chilton are...
All hail Brack!!!!
Bosses (Score:5, Insightful)
Re: (Score:3, Funny)
Apparently the programmer's boss is also a programmer, the artists boss an artist and they are expected to work together. So so SOOOO much better than the bureaucrats most of us get stuck with.
Yeah. You say that now. Then you'll get a job where your boss is a programmer, and it'll be like "Why haven't you finished that task yet? I could have done that in 2 hours, and you've been 6...", and no matter how much you argue about how long such a task takes, you'll never win, because he'll _know_ exactly where [xkcd.com]
Imagine what these guys are like... (Score:4, Funny)
I wonder what those two guys are like. I'm pretty sure they must be nerds of EPIC proportions. And I don't mean that in a mean way, I'm just sayin....
Ob. ctrl-alt-del (Score:4, Funny)
How it REALLY happens behind closed doors:
http://ctrlaltdel-online.com/comic.php?d=20090916 [ctrlaltdel-online.com]
Re: (Score:2)
Not much detailing about their data centers. It seemed more of a Blizzard PR piece than anything else, and without technical details, this is just another Blizzard ad.
"The PR and community teams were Brack's next focus, the groups responsible for public interaction. The PR team has helped to ensure some 10,000 articles have been written about World of Warcraft"
Job well done, PR team!