Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
Role Playing (Games)

WoW Downtime Interview at Penny Arcade 149

Posted by Zonk from the called-out-and-questions-answered dept.
Last week, Tycho mentioned a set of questions they'd sent in the direction of the folks at Blizzard regarding the downtime World of Warcraft has been experiencing. Today Blizzard responds to their questions. Tycho agreed to the interview being reposted below the cut, so read on if you'd like Blizzard's response in the face of player frustration.
1. You say that you sold six hundred thousand units. Is the game not capable of supporting this many users?

The short answer is "The game is capable of supporting this many players," but it would probably be helpful to provide some background information. Based on our market analysis, we made some initial calculations about the size of the massively multiplayer online games market in the United States. We then accounted for new customers to the genre based on our previous games. Looking over this data, we did believe that there was the potential for an extremely sizable interest in a Blizzard MMOG. According to our research, other successful MMOGs in the U.S. had achieved roughly 300,000 subscribers after 12 months of operation. What ended up happening with World of Warcraft is that we achieved double these numbers in approximately the first six weeks of launch. We absolutely can support the number of copies we put on shelves, but we believed it would take us longer to get to this number in terms of players purchasing the game and logging on.

We had not anticipated this amount of growth in such a short time; however, we did have a backup plan that was deployed rapidly. In the first week of launch, we more than doubled our number of game servers and server infrastructure to accommodate the demand. The fact that we had planned to grow the service over the first 12 months of operation was evident, as we had server hardware waiting to be deployed. We just anticipated that this server rollout would be gradual. Copies of the game were being purchased at a much faster rate than anticipated, so we had to abandon our slower-paced plan and go into rapid deployment to accommodate these additional customers. This meant we also had to advance our timetable for additional server purchases.

With such a rapid growth of the network, we started to see several bottlenecks in the infrastructure that exposed themselves very quickly when the expanded hardware immediately took on massive load. These bottlenecks were solvable, but they required additional upgrades to the backend systems to accommodate the load--which, again, we hadn't planned to see, even with the extreme estimates, until later in the year. Regardless, server stability has remained our number-one priority, and so we acquired and deployed even more equipment as part of the process of addressing these issues. All of this new hardware also required additional software and operating system upgrades on the backend. The problems that some players on the 20 or so most populated servers (out of the current total of 88 servers) have been experiencing are related to some of the upgrades not functioning as desired. We are working diligently with our vendors and internal technical staff to get as quick of a resolution to the problems as possible, and we believe there should be noticeable improvements soon. When our community team commented that people are working 24/7, they weren't exaggerating.

2. If it's true that the server problems are related to the overwhelming number of players, why was no effort made to better distribute players evenly across realms, or allow players and guilds to transfer to less populated servers?

We actually did have a number of checks in place at launch to distribute players as evenly as possible across realms. When a new account logged in, the game would ask what realm rule set and time zone the player preferred, and then it would suggest the realm with the lowest population that matched the selected preferences. That said, we're definitely working on resolving the overpopulation problems that ended up occurring on some realms despite our preventative measures. A realm-transfer option that would allow players to move from their high-population realm to one with a low population is one of the things we're investigating. We're exploring this option fully and hope to be able to communicate more detailed information about it to our customers in the coming weeks.

3. Currently, large scale player raids involving large groups of players experience a huge amount of latency. How do you plan to compensate for this in your upcoming PvP Battlegrounds feature?

The player raids often have hundreds of people per side in one area; that area is on a server that is also running the rest of the continent, and that can result in the latency you describe--depending, as well, on the total population of that server. We're continuing to look into the issues surrounding this dip in performance. Battlegrounds, on the other hand, will run on the instance server, so there should be no such issues. Additionally, players will be unable to "zerg" in Battlegrounds; there will be a limit to the number of players per side.

4. What accounts for the frequent "emergency" maintenance downtime? What issues are you attempting to resolve?

The emergency maintenance periods are to restore stability while we continue to narrow down the cause of the problems. Some of them are also to deploy temporary fixes to various in-game systems while we continue to develop a longer term, more stable solution. World of Warcraft delivers many complex features that are unique to MMOGs. Features such as the in-game mail system, auction houses, player inventories, flight paths, quest states, etc. use a lot of server bandwidth, which makes pinpointing problems on the server infrastructure much more complicated.

Recently, the extended emergency downtime for a certain number of realms was needed in order to better accommodate our growing player base. Some of the upgrades that we planned for all of the realms were made to these realms first, as they are among the most populated and thus most in need of aid. We set the realms up on the latest top-of-the-line hardware and made the software upgrades accordingly, but some unforeseen issues cropped up with the database that resulted in the problems players currently see. This is no fun for our player base, of course, and we don't want to keep the realms running in a condition that frustrates our customers when we can attempt to fix things . So, these downtimes have been used to change hardware and apply fixes that will hopefully alleviate the issues. We have not yet resolved the problems, but we're working on this around the clock.

5. What issues are you experiencing with your login/authentication servers? It is often the case for myself and the people I play with that we cannot access realms our friends are already logged into.

These types of issues stem from the problems described above. Conflicts occur between some of the internal applications running in the background, and the end result can take the form of temporary login issues. We're working to resolve these conflicts so that they are no longer a factor.

6. When do you expect to have the worst of these problems resolved?

We'll be constantly working on these issues each day moving forward until they're resolved, but we don't currently have a set date for when that will be. We're doing all we can to make sure these problems no longer occur -- it's our top priority, and we hope to have the issues fixed as soon as possible. We'll continue to provide players with regular updates on our progress.

7. Will the European launch utilize the same realms, or will these players be hosted on all new equipment? If they are hosted on new servers, what have you done to ensure that the launch will be free of the problems mentioned above?

They will be on their own set of hardware, as with our Korean release. Our teams are learning from the experience of our North American launch and are applying that knowledge to the servers in Europe. We hope to provide them with a smooth launch.

8. What would you have done differently?

It would be easy to speculate about what we could have done differently, but that wouldn't turn back the clock. Right now we're extremely focused on the issues at hand, and this focus is helping us methodically chase down the problems that are causing frustration for some of our players. The foundation of our company is based on providing a top-notch game experience and an equally top-notch level of customer satisfaction; we won't be happy until we feel we're consistently meeting those standards.

This discussion has been archived. No new comments can be posted.

WoW Downtime Interview at Penny Arcade More

WoW Downtime Interview at Penny Arcade

Comments Filter:
  • Bout time they got round to this. Kudos for Blizz. for using the word Zerg in something about WoW. Freudian slip?

    • Re:Bout time. (Score:4, Informative)

      by meta-monkey ( 321000 ) * on Monday January 24, 2005 @03:14PM (#11459760) Journal
      No, it's commonly used by players, as well. It means you get rushed by a bunch of players. If this were the spelling bee, I'd say:

      Nation of Origin: English, (more likely Korean)
      Definition: To be rushed by several players.
      Use it in a sentence: "So we were attacking Tarren Mill in Hillsbrad when the hordies called in all their guild mates and zerged us."

    • Re:Bout time. (Score:2, Informative)

      by Anonymous Coward
      Nope, "zerging" has become a gaming term for massive assaults that overwhelm the opponent by shear numbers.
    • Not a slip. Zerging is a real MMO term nowadays. DAoC players tend to use the phrase a lot.
    • Zerglings are a race in the game Starcraft (by Blizzard) The predominant strategy in playing them (in simplified terms) is to make a shitload of them and send them against your opponents. Hence the terms 'Zerg' or 'Zerging'

  • WoW section (Score:1, Funny)

    by Anonymous Coward
    In addition to games, politics, yro, etc it seems there should be a WoW section, surely it would be more popular than other sections, e.g. BSD, for as we all know BSD is dying.

  • MMORPG Players.... (Score:5, Insightful)

    by m0rph3us0 ( 549631 ) on Monday January 24, 2005 @03:15PM (#11459796)
    The people at Penny Arcade act like they are investigating a space shuttle disaster. It would be nice to see what level of respect the people who play games 24/7 and actually care about intermittent downtime would give a customer paying them $14 / month. I bet most of them would tell the customer to "F off you n00b".
    It is a $14 a month service for unlimited entertainment, you can't expect that every single kink will be ironed out at launch.

    Real world devices have real world problems, and a whole host of gamers, like Tycho fail to realize that.
    Imagine if Tycho had to deal with 100,000's of people complaining that one stroke in one particular comic was 1 pt off.

    The problem is that if Blizzard put the resources into making the game so that there were no problems at launch and that they had the server infrastructure to support the entire planet logging into same screen all at once the subscription fee would be so incredibly expensive that no one would play the game.

    Manufacturing defects are a trade off, yes Blizzard could build a game with no bugs, but how many players would want to pay $5,000 for a copy, and $1,000 a month?

    • Seriously,

      Judging by how often www.pennyarcade.com goes down, you'd think Tycho and Gabe would know that technology has its problems. Remember during Child's Play last month after they were mentioned in all the mainstream news sources? Slashdot actually had to host the comic here so that PA's servers could take a pounding from all the non-regulars.

      I think Tycho needs to chill the f*ck out and play something else for a while.
    • Dude... it's not unlimited entertainment. If it's unlimited, then you pay once $50 game and play it unlimited number of times. This is a subscription service based game at $14 a month.

      Blizzard needs to put more money into more servers. Management is simply being cheap. Remember, this is the same company that fired the Diablo II team after the success it had.

      • Throwing more money at a problem may "fix" the problem, but you'll never learn about how to truly fix the problem unless the problem happens.

        Granted, they could throw money at "more servers" at this point to alleviate the problem (which they are doing) but if they learn what the true cause of the problem is, then they can truly fix it and not have the problem crop up again when more people sign up....

    • Re:MMORPG Players.... (Score:5, Insightful)

      by p7 ( 245321 ) on Monday January 24, 2005 @03:48PM (#11460290)
      First off, it is not unlimited access. Your monthly fee only gets you 1 month of access. Second some of us can't play 24/7. We have to work and sleep. So I am completely within my rights to complain if the service that I paid for is not available when I want to use it. Internet access is in the same ballpark price wise. Would you accept lots of downtime with your ISP? Or would you move to an ISP that gave you waht you wanted. There is a lot of competition out there and Blizzard is providing a service, they need to make the user happy otherwise we will find an alternative. I don't expect all the kinks to be ironned out at release. I expect to be able to use the product and not pay for significant down time.

      I personally commend Penny Arcade for bringing this up.
    • It is a $14 a month service for unlimited entertainment, you can't expect that every single kink will be ironed out at launch.

      And why, oh why, out of the many services that we pay for and hold to a standard: Mobile phones, home phones, Internet, Cable TV, etc. do we not hold MMO's out to a similar standard?

      Because it's on the Internet and seen as a waste of time? I'm sorry, but is it a fake $14 that is being paid every month? Are they not held accountable by the quality of their service like every other

      • Re:MMORPG Players.... (Score:4, Insightful)

        by flibuste ( 523578 ) on Monday January 24, 2005 @04:59PM (#11461367)

        And why, oh why, out of the many services that we pay for and hold to a standard: Mobile phones, home phones, Internet, Cable TV, etc. do we not hold MMO's out to a similar standard?

        Oh..because you never had trouble with cell phone or home phone companies?

        I mean:
        * Getting overcharged or having billing issues (which seem to be the common denominator in all phone companies). This happens ALL the time. So much that it's nearly fun to receive a new bill and find out what blling part is screwed up for the last month.
        * Not have the cell phone working where you expect it to work (bad coverage).
        * Phones advertised with a lot of features, but you find out it's just lies since features are capped by companies so you overpay them for idiocies like sending images (as an example, TELUS is blocking their phones so the non-geek cannot use a computer to get images from the phone). * Cable TV repair guy coming 1 week after the scheduled date (whicho of course was inconvenient enough that you already had to take a day off from work).

        To me Blizzard is actually doing pretty well in working toward customer satisfaction. All this whining is nonsensical and your arguments just don't hold a small exam.

        At least Blizzard doesn't charge you for accessing customer support where I think it is EXTREMELY SHOCKING that in North America, phone companies charge a monhtly fee for access to 911, whatever the fee is.

        Why is it a game that 'people spend too long on' is any different than a mobile phone service used for communicating with friends, where days worth of downtime a month would be considered terrible service?

        Because it is a GAME, not a business or life-threatening *service*. It is not even a *service* as such.

        Honestly, this game is working great with a few post-launch glitches due to unexpected success. Have you ever lived the launch of pathetic "Ultima Online"? In constant value, it was more expensive than WOW is at that time. And my! What a load of crap it was! Funnily enough nobody seems to remember that mess.

        So give us a break and quit the whining

    • Re:MMORPG Players.... (Score:4, Informative)

      by neura ( 675378 ) on Monday January 24, 2005 @04:18PM (#11460760)
      You're acting like this is just a single customer out of the many that is interrogating Blizzard, as if to ask "how would you like it if every customer did this to you for $14/month?". Correct me if I'm wrong on that. In response though, this is not just a customer, this is a well known site that a lot of people get a good dose of their gaming information from. Blizzard knows this and hence why they have responded at length and with an air of (sometimes painful) honesty.

      This has already been addressed in other replies, but I'll state it again here. Yes, real world devices have real world problems, and when real world services go down, people get pissed off, switch services, ask for refunds, reimbursements, pro-rated bills, etc.

      100,00's of people complaining that one stroke in a particular comic was 1 pt off would be like 100,000's of people complaining that a building in the game was misaligned slightly, but still compeltely usable. What people are really complaining about in reference to Blizzard and WoW is not being able to access the service at all... like if people were PAYING to see Penny Arcade comics and then the site was down all the time and people couldn't see the comics or the site would be up, but viewing the comics required a login and it rejected all paying customers.

      If you had bothered to read the article this post is referring to (which you obviously can't, with your head so far up your ass), you'd know that the problem had nothing to do with not spending enough money on the server infrastructure and more to do with Blizzard's extensive market research showing that even in very generous, extreme estimates, the number of people signed up to play this game would be less than HALF of what it really turned out to be.

      Again, I just want to stress that we're not talking about annoying software bugs here, we're talking about not being able to use the service they are paying for. It's like buying an operating system that has a 50/50 chance of booting. >.> ...or having a phone line that you might be able to call someone on... once in a while. If this were just a case of bugs like, an NPC not giving you the quest you need or items not working in game, but you could still play the game and enjoy the rest of it... I don't think people would complain 1/10th as much as they are.

      Personally, I LOVE this game and like Tycho's previous posts about the game, the only thing that makes me really really mad is that I can't play! I have NO intention of returning the game. I just want the server instability (putting it lightly) to go away, so I can enjoy everything else about the game.

      Access to a product that you pay a sign up fee and monthly fees for is not a priveledge, but part of the contract you enter into when you agree to pay them every month for the service they have told you you were getting for that monthly fee.

    • Re:MMORPG Players.... (Score:5, Insightful)

      by meta-monkey ( 321000 ) * on Monday January 24, 2005 @04:18PM (#11460764) Journal
      I would imagine the casual users would be more pissed off than the 24/7 guys. Some days I have like 5 hours to play, so I play for 4 hours, and then it gets wonky, so I take that as my cue to go do something else. However, if I worked all day at my job, came home and spent time with my family, and then wanted to play for an hour after the kids are asleep only to find I can't login or the game is laggy as hell, I'd be pretty pissed off. what are you paying your $14/month for if you can't play during the limited time you have available?

      • Re:MMORPG Players.... (Score:2, Insightful)

        by Yer Mom ( 78107 )
        Especially since most of the users in that category are going to be logging in at around the same time (say 9pm-ish) which makes it more likely that they're going to get hit by the problems on a regular basis.

        I work Mon-Fri, 9-5. By the time I get home and eat, it's about 6:30. Saying "yes, it may be down from 6pm onwards, but it's fine during the day - play then!" wouldn't pacify me one little bit, because that's no use to me.

    • It is a $14 a month service for unlimited entertainment, you can't expect that every single kink will be ironed out at launch.

      Yeah, the gall of people to expect a game has gone gold to 'work as advertised', and 'be available to them' after they pay 50 bucks + 15/month for it.

      Imagine if Tycho had to deal with 100,000's of people complaining that one stroke in one particular comic was 1 pt off.

      I'm sure it happens to him most times the strip isn't up first thing in the AM. the obvious difference, of
    • It is a $14 a month service for unlimited entertainment,

      Holy shit! Where do I sign up for unlimited entertainment a month?!

      Last time I checked, you still had to walk/ride a boat/fly to your destination, you still had to spend time doing menial quests only to recieve a shitty reward and no matter how many days I play without sleep theres always some asshole out there with better gear than me and will rub it in my face. My boss at work does that to me and I get PAID to do it. I'm sure as hell not going to P

  • This is not good... (Score:3, Insightful)

    by keiferb ( 267153 ) on Monday January 24, 2005 @03:16PM (#11459813) Homepage
    Blizzard actually made an effort to let the community know what was going on... Now the forum trolls will have to find something else to whine about. Something tells me it won't be pretty.

    • Re:This is not good... (Score:3, Insightful)

      by JVert ( 578547 )
      I'll bite.

      It just seems more blatently obvious that they should give out free month. The people spoke and blew away the companys expectations within a week bringing in what they anticipated to take 6 months. For this the players are punished. Battlegrounds has to be taking a back seat untill they solve the server issues.

      I'ts not like i'm going to quit playing because I wont get a free month. I can't even bring myself to try the free Anarchy Online. I'm just offended personally that they belive that the cu
    • They've been making an effort all along... they just don't have the best communication between the people who actually know what's going on and the community reps, so the reps can only keep posting things like "We know, we're working on it". That's what gets the players all frazzled, because it's the same line every time. While this one doesn't go into many technical specifics, it's much, much more than what we've been given to date.

      Also, as they've stated several times, the engineers in charge of gettin

    • Re:This is not good... (Score:5, Insightful)

      by Ayaress ( 662020 ) on Monday January 24, 2005 @06:19PM (#11462326) Journal
      Having been an MMO play for a long time, they won't have to find something new to complain about. A few examples (Copy/pastes from the Ashen Empires forums, mind you):

      Example 1 (Context: SoE is the Staff of Enervation, the most powerful mage staff in the game. It's magical wood, meaning it can be repaired by a Mind based repair spell. A database error during the shift from TKO Software back to private ownership made it Nature repair - which is for magic wood - a few weeks ago, and that was corrected shortly tehreafter. This post came a while after that)

      WTF SOE IS MIND REPAIR NOW???
      by (removed)
      YESTERDA YIT WAS NATURE REPAIR WTF U CHANGE IT FOR?

      by (removed)
      I sure hope not that's gay I don't want to have another skill. Well not really I have 100 in all mage skills but its gay because I need to use my mind runes now.

      by (removed)
      WTF FIX THE FUCKING BUGS BEFORE YOU POST A NEW UPDATE

      Another example:(Context: New land area was added recently, which is only suitable for parties of high level characters. A level 15 should not be there for any reason. The area is open, and is accessible from both lawful and criminal cities, although the walk from Desprail or Redwake makes it hard on criminals, but just about anybody can get there if they want)

      Re: WHY do i see CRIMINALS running free in the new lands?
      by (removed)
      what city? there's no city, just a lot of grass and big ogre things that kill me. why the hell do the monsters so fucking high level? hows a level 15 supposed to bow in there geeze its not fare nemore

      The following posts are by the same person, less than four hours apart: (Context: the update was scheduled for the day before, but was delayed due to a bug)

      Where's the UPDATE????
      by (removed)
      Just put it on the servers so we can play it and we'll find the bugs stop slacking we pay your salary.

      Wtf's with the fucking BUGS!?
      by (removed)
      FIX THE FUCKING BUGS before you post a patch. you should NEVER post any update unless it is ONE HUNDRED PERSENT BUG FREE goddamn incompetents. you can't run a fucking game. We pay your salary now do your jobs.

      Frankly, in this game, I consider this sort of brattyness worse. The game is fairly small, and has been struggling to pay the bills its whole existence. Twice, they licensed larger firms (most recently TKO Software), and both times they got screwed. The first time, the developers broke the contract, gave up ALL the money the game had made for nearly a year, and went without a lot (Lead developer even sold his car) to keep the game alive. The second time, it was a bit worse. They all got canned by the new owner of the game, and the game was going to be shut down. They stepped in and paid - out of their own money - to license the game back and keep it alive. For more than half of the last two years, the game has been free for one reason or another, and for the rest, it was only $8 a month.

      Granted Blizzard doesn't have these woes, but people will still complain about the exact same things. Even if they're shown that the problems are being fixed, it won't be enough, because it's not being fixed instantaneously.

      Once, AE had a very strange bug. It was very difficult to reproduce, and the circumstances that caused it were never properly pinned down. Less than three hours after it was found (At 4 AM yet, the devs were probably asleep), people were already complaining about why there hadn't been an update to fix it. When there was an update, they complained that the server was down for almost "FIFTEEN MINUTES OMG I R WANT MI MONIE BAK."
  • ...I wonder why they never moved beyond the "realm" concept and put together a single, huge, continuous world (where different areas would be physically located on separate servers, even with several servers serving a single area or sharing area serving wherever needed).
    As far as I know, the only MMOGame that ever attempted that approach was EVE-Online, and they have a record of a bit over 10,000 clients logged on and playing at the same time.

    Personally, I find the entire concept of realm "sharding" to be

    • Re:And again realms and servers... (Score:1, Informative)

      by Anonymous Coward
      WWII Online

      Check em out [wwiionline.com]
    • As far as I know, the only MMOGame that ever attempted that approach was EVE-Online
      Indeed, and it works pretty well. There's an avg 8,000 players on at any given time, however there seems like there is alot of lag (if you would even call it that). It takes about a second to respond when you do anything. Not sure if it's a server problem or anything, but it may show up in other MMOs that try a single world setup.
    • Ragnarok Online uses the same mechanism. The overall map is split up into large squares. You walk off one square and you go to a different server when you appear on the next square.
      • Ragnarok Online uses the same mechanism. The overall map is split up into large squares. You walk off one square and you go to a different server when you appear on the next square.


        Though it's not a "game" in the traditional sense, Second Life does the exact same thing.
    • Second Life uses this same one-big-world-with-different-sections-being-differ ent-servers approach. Like when you fly from one section into the next you're actually passing onto a different server at the same time. It seems to work OK, unless everyone is trying to get to one particular piece of property...then it can slow things down a bit.
    • I wonder why they never moved beyond the "realm" concept and put together a single, huge, continuous world (where different areas would be physically located on separate servers, even with several servers serving a single area or sharing area serving wherever needed).

      Have you played this game? The AH in Ironforge is packed on my server, and it is one of the low-population servers. I have seen screenshots of it from a high-population server. Can you imagine what it would be like if there was only one s

      • The auction houses are a completely different problem though. They way they have it setup is stupid. There is no reason to SEE everyone in the house when you walk in. It should look empty to everyone as soon as you enter the building. You should only be able to interact with the auction system inside an auction house. You should also be able to access the auction system from any inn in the game.

        Auction houses should be treated like a small instance.
        • I completely agree with you. AH as an instance is a great concept and would spread the load evenly. Problem I see is that Blizzard hasn't yet fixed some of the instance problems that exist; some people are reporting bugs from instances like getting 2.5levels of rest XP and it seems to be tied into instances from what I recall. Also as instances are loaded on different servers it could create DB or network problems, depending on how the DB is setup. So instancing an AH may make things worse, but it depends o
          • I wouldn't instance the db part of the auction house, just what the player actually sees. Or instead of using an instance just make the actual auction house a different kind of area. When you enter you don't see anyone but you and the NPC to interact with.

            Blizzard has stated that they aren't going to do this because they feel it changes the feel of the game in that entering a building with a ton of people makes it feel more alive. Thats fine. If they don't want to change that then they need to add mult
      • The AH in Ironforge is packed on my server,

        That's an easy thing to address;

        You could provide more locations to access it. Why don't Stormwind and Darnassus have auction houses that are connected to the one in Ironforge?

        Or you could move it to it's own landblock, off-loading it onto separate hardware like Asheron's Call did. That would also benefit the areas around it.
        • There are 88 servers. Divide the auction house crowd by 3, then multiply by 88, and you've still got 30x the load in an AH. More like 40x, since one place will always be more popular. And the people come for the AH but they're spread all around the city, so the whole city is crowded. The server load isn't proportional to the number of people all in close proximity. Its proportional to the number of people in close proximity squared, because each person has to be aware of each other person. The whole g
    • The problem is, what do you do with the 20,000 people who all want to be in the auction house at the same time? Even with 88 servers, the one on my server has 200+ people in it at a time during prime time.
    • Its not even that big of an issue of hardware but gameplay. There are alot of servers in the game right now, ALOT. If you had to combine all the players together your land would be very camped and have a huge affect on gameplay/economy. It was determined in advance that the world would be designed with an ideal amount of players for the purpose of gameplay. Whats interesting right now is how the economies are different across different realms but they are still faily balanced even when it changes. On a mas
    • That wouldn't work.

      I don't know if you've played WoW at all, but in the beta I quite often had problems with areas being just too damned crowded. In some areas (e.g. Westfall), the mobs you had to kill for your quests were always dead already, with several players camping their spawn points. And then there's contested territory. Horde player doing Tarren Mill on a crowded PvP server? Good luck staying alive more than two minutes at a time!

      That was with about 1200-1500 players logged in to the realm, I
      • The crowding of the quest areas isn't that big a deal anymore, now that the players are so spread out. I bought the game on day one and of course the quest areas were swamped. Everybody in the game was the same level, and all doing the same quests. Now that people have had a chance to spread out and diversify, I don't really see this problem anymore. I created an alt and blew through the first 20 levels MUCH faster than the last time, because I was one of the few players in the areas I needed.
        • Depends. The problems I had with Westfall were well after the players had a chance to spread out. On the other hand, there were areas meant for the same level players which were pretty empty, even then. I did some questing in Thelsamar and saw very few people. I suppose this is because dwarves and gnomes are unpopular as player characters, and the Human and Night Elf masses are mostly too dumb to think of questing in another race's home region.

          Still, with 50x the players on the server, it would surely
    • That approach won't work for a lot of games. In these games, people usually don't try to go off and find a place of their own, they go to the popular places and farm the mobs everyone else is farming. In many cases this is because certain spots are genuinely better for gaining experience or getting cash or items (and it's difficult for the developers to prevent any spots from being better than the others without making the game bland). People will even wait in line to get into a group camping a really good
        • Having a massive game world running on a large cluster of servers doesn't help anything if your players only inhabit a fraction of them.
        I think your post is spot-on, but I think the proper summary is: it's a game-mechanics issue, not a technology issue. There are several cases where it'd be not-fun to have 10,000 people in the same place (even if graphics cards were advanced enough to render that without breaking a sweat).
    • The workload scales exponentially. Even with some wicked parallel code running on computing clusters the problem is intractable - you can support 10k people on 10 server racks, or 100k people on 200 server racks. Since the 10x increase in people requires 20x the servers it makes more sense to split things up.

      There is a balance here.

      As processing power becomes cheaper this problem will ease, but in the end it's still O(n) hardware trying to solve an O(n^x) problem.

      If they could distribute the proc
      • Hmm. Distribute processing to the users. This is interesting. So combine something like bittorrent with a MM game. Each client uses their own bandwidth to support the whole. Doesn't sound right, but sounds interesting.
        • Unfortunately, it's virtually impossible, as every multiplayer game has painfully discovered at some point. You can't have the client doing ANY meaningful processing and maintain the integrity of the game, because the client WILL be hacked to death by people looking for an advantage. EVERYTHING has to be verified server-side, and you can't send ANY information to the client without assuming that the player is going to see it.

          Peer-to-peer networking games like Starcraft, Diablo (1), Warcraft II (are we sens
          • The first thing to consider here is a SINGLE server (or server group) that handles/stores user login info, character data, equipment/inventory, etc. Whenever you login to the game, wherever you chose to play, you only have a single login name/pass and you always have the same gear. The "security" issue would therefore be limited to internal communication (between the login/auth server/servers and the game-area server you play on).

            Now, wether you pick the option of a HUGE world with zilions of players insid
            • What type of crack are you on? The basic issue is that people tend to crowd. In each and every case. So you must have each area server be able to handle at large proportion of the populace. So it doesn't scale well. Sharding is the easiest and simpliest solution. The suggestiosn you made woudl ruin any game. Make all distances longer? yeah walking for 2h is what I consider to be fun!

              Even iff all issues with mobs are fixed and every mob can drop good and useful things, and each city has their own server, wh

  • Perl sure can be tough to debug (Score:5, Funny)

    by snuf23 ( 182335 ) on Monday January 24, 2005 @03:18PM (#11459834)
    "Conflicts occur between some of the internal applications running in the background, and the end result can take the form of temporary login issues."

    Translation: Bob's nifty 12 line perl hack doesn't seem to scale.

  • Nicely Spun answers (Score:1, Insightful)

    by Anonymous Coward
    They shipped an estimated 2 years worth of product at release? Even if you expected to be more popular than the other MMORPG, it seems silly that you would create so much, when it would be considerate to remaster the game at some point so that new users aren't forced to download a huge patch.

    It sounds like the answers are spin for we weren't ready to handle a game of this scale. We maybe soon, but you all are going to have to put up with us through the learning process.

    By the way they had to release e
    • I suspect it's likely cheaper to stamp out a product all at once than constantly returning to the manufacturer. Since the game needed updating the day it was released (the beta client was more updated!), it wouldn't make too much of a difference to Blizzard how new the disks were. Besides, I'm sure you've heard of their much-vaunted torrent updater, so bandwidth costs would be relatively low compared to other games anyhow.

      Oh, and it was one year - I think they'd probably have boxes sitting around in a wa
    • Actually, they didn't print all 600,000 copies at release. They printed around 200,000 if memory serves, and expected that to last until early 2005.

      Those 200,000 sold out almost instantly (do you remember the problems around Thanksgiving trying to find a copy?), and the publisher Vivendi printed the excess more and sold them. Although its Blizzard's product, keep in mind that the publishers really drive the sales of a product. Blizzard asked Vivendi repeatedly to stop printing new copies until they had

  • Not surprising (Score:2, Insightful)

    by Bigthecat ( 678093 )
    It's great that they responded and all; but I have no idea why I keep expecting something different from an MMO company response when we always get the same stuff.

    It's what we already knew: They 'could' have supported many players, they 'tried' to evenly distribute players and they 'will' fix the problems at some unknown date.

    I'm not sure what I was expecting, but a watered-down version of what we already know that doesn't cast any blame in the slightest wasn't it. Where is that 'Blizzard Difference' ever

  • Allow me to translate (Score:5, Funny)

    by PoderOmega ( 677170 ) on Monday January 24, 2005 @04:17PM (#11460746)
    1. You say that you sold six hundred thousand units. Is the game not capable of supporting this many users?
    No it isn't. We had an untested backup plan that did not work either.

    2. If it's true that the server problems are related to the overwhelming number of players, why was no effort made to better distribute players evenly across realms, or allow players and guilds to transfer to less populated servers?
    We recommended a lower population server, but did not explain the implications of the choice, so people just made uninformed desicions. (Note: I would take a higher populated server if I didn't know it meant more lag. Do you want to play in a void??)

    3. Currently, large scale player raids involving large groups of players experience a huge amount of latency. How do you plan to compensate for this in your upcoming PvP Battlegrounds feature?
    Battlegrounds won't lag because we limit the number of people in a battle.

    4. What accounts for the frequent "emergency" maintenance downtime? What issues are you attempting to resolve?
    Once again we did not compensate for the high number of users. Do you think we had 600k people in our test environment?

    5. What issues are you experiencing with your login/authentication servers? It is often the case for myself and the people I play with that we cannot access realms our friends are already logged into.
    Once again we did not compensate for the high number of users. Do you think we had 600k people in our test environment?

    6. When do you expect to have the worst of these problems resolved?
    When enough people get pissed and leave and we finally upgrade. So in about 6 months.

    7. Will the European launch utilize the same realms, or will these players be hosted on all new equipment? If they are hosted on new servers, what have you done to ensure that the launch will be free of the problems mentioned above?
    We're not sure.

    8. What would you have done differently?
    Better market research, I guess. We'll get it fixed, don't worry.
    • Gee, a critical response in a discussion - (You know, the other side of the argument?) gets modded flamebait because the moderator doesn't agree with the opinion. Such a surprise.
    • This is not flamebait. It is a critical analysis of responses laden with evasive language. The game might be capable of handling all the players but they did not plan correctly and blew it. The poster's comment on #2 is DEAD ON. #6 is probable. #7 is accurate. #8 is a great summary as well, somehow blizzard failed to realize that WoW was just about the most anticipated game of the year, maybe after Halo 2. "Here is the pulse, and here is your finger, far from the pulse, shoved straight up your ass."
      • Uh, 7 is clearly wrong. Unless the interview has changed since I read it on penny-arcade.com, answer 7 explicitly states that each region will have its own set of servers, and that the teams setting up these servers are learning from the mistakes of the American team. (And benefiting from their patches, no doubt.)
    • Actually, to me it sounds like a BattleGround will be a fully separate server, conventional online-game style. So it sounds like its more than just capping the players - its like switching from playing an WoW to playing UT2k4 with your WoW character. Which sounds cool.
      • Well, the way WoW's servers are set up is kind of odd, each "shard" is actually three seperate pieces of hardware. There's one for Kalimdor, one for the Eastern Kingdoms, and one for all the instance dungeons. Battlegrounds will be run on the instance server. This is also why you have a load screen going into places like the stockade.... it's not so much to load the textures and geometry as to cover up the fact that you're jumping servers.

  • 500,000 beta accounts? (Score:2, Insightful)

    by Xlipse ( 669697 )
    I thought WoW had like 500K beta accounts/testers?

    The bean counters were either sleeping or stupid.

    How could they NOT have seen this coming?

    Are they are full of BS?

    or more likely:

    This is an example of a large corporation with very slow turning wheels when it comes to planning/changing plans.

    I give them credit for having a backup plan in the first place... but personally, I expect MORE of Blizzard. So many beta accounts should have been an obvious indicator for what was in store, I would think..?
    • Well, they did have around that many in OPEN beta, but OB only lasted for little over a week. The closed beta period lasted much much longer with nowhere near that many participants. Basically, it seems like they were only concerned with heavy stress testing enough to make sure the servers didn't melt thru the floor and decided to leave the scaling issues for later. This was probably put off to meet a christmas ship date.

      In their defense, you can't realistically go into a release/new business EXPECTING
  • Now someone needs to do an interview with Tycho and Gabe about why the penny-arcade.com forums can't seem to stay available for longer than fifteen minutes at a time.
    • When you have to pay for PA forums, that will be an interesting question. Maybe /. subscribers should ask when /. will generate valid HTML. That's one of the biggest reasons I won't subscribe, though the biggest is the adblock extension and there's also the issue of the total lack of editing skills among the so-called editors.

  • 7. Will the European launch utilize the same realms ?

    They will be on their own set of hardware, as with our Korean release. Our teams are learning from the experience of our North American launch and are applying that knowledge to the servers in Europe. We hope to provide them with a smooth launch.

    Does this means people in north america won't be able to play with our europeans friends?

    • Re:Europe and NA (Score:3, Informative)

      by Quikah ( 14419 )
      Yes. This has been known about for a long time, ever since the beta. If you want to play with folks in Europe you need to either buy a European copy which connects to the Europe server or have your friends in Europe buy a NA copy which will connect to the server in NA.
  • This reminds me of the Star Wars: Galaxies roll out on June 26th, 2003 (if I remember the date correctly). I had my copy purchased, and installed, patched, and clicked on that 'Connect' button, and all of the servers were offline. Every single one. Apparently they severely underestimated the number of players who would be logging in that first day, and for some reason, nobody could.

    Their answer? They gave folks 1 extra day of play time before their subscriptions had to be renewed.

    People were complaining a
  • Come on - when was the last time blizzard pulled off a successful launch? Delays, bad code, bugs - the fact of the matter is that they produce a superior product - 6 mos later its bug free. I personally find their dev style refreshing - they dont cave to demands like so many companies, and they don't put things off indefinatly - ceps ghost...

    This is blizz's learning curve - they've never made a MMORPG before. Who in God's name could have predicted this kind of response? 600,000 people!!!

    Bottom line - A

  • I think Blizzard got a double whammy of success - 600,000 new players in two months and players loving the game so much that they play every night.

    One thing that Blizzard touted early was the ability to play it casually soloing - you can play once a week and easily gain levels doing quests. By following the flow of quests, you can often do two in an hour. However, that rapid pace of advancement makes the game fun in general. You get rewarded with new skills, new environments and new challenges constantly.

  • Hats off to Blizzard (Score:3, Insightful)

    by chrisbro ( 207935 ) on Monday January 24, 2005 @06:31PM (#11462467)
    I can't think of a better way to handle the situation than Blizzard has so far. They're a business, not a Entity of Gaming Awesomeness that has a crystal ball to see market demand in the future. They did some research, saw the numbers, and bought the number of servers that they thought they'd need. When a problem came up, they let everyone know, allowed another free month to some, are working hard to fix the problem, while still keeping in touch with the public with what's going on.

    What more do people want? They have tried their best based on what they know, and when things went wrong, they responded very quickly. Several MMOs have problems and are dickheads about it.

    The fact that Penny Arcade yanked the Game of the Year award away from WoW is just immature, in my opinion, given Blizzard's response to the situation.

    By the way, I've been noticing that Penny Arcade takes a shitton of time to load. I demand they fix the problem instantaneously! I don't care what it takes! I'm going to yank their links from my site unless they get on it right now! I'm sending "terse questions" next week.
    • "By the way, I've been noticing that Penny Arcade takes a shitton of time to load. I demand they fix the problem instantaneously! I don't care what it takes! I'm going to yank their links from my site unless they get on it right now! I'm sending "terse questions" next week."

      You've, of course, sent your $15 to Penny Arcade this month, too, right?

  • It's a GREAT basic design (Score:3, Insightful)

    by Malor ( 3658 ) on Monday January 24, 2005 @07:28PM (#11463076) Journal
    One thing that people don't often mention is that, while the lag was getting pretty terrible, the game was still mostly playable. It might take several minutes(!) for an Auction House query to return, or for an auction to be created, or for your email to show up, but the transactions DID eventually happen.

    Instead of just blowing up completely, by and large, WOW fails fairly gracefully. The engineering in that is non-trivial. I don't think people realize just how good that code is. Getting a system that stays reasonably stable under completely untested loads is really, really hard. I am VERY impressed with the quality of their design and code work.

    And, yes, some data did get lost... the servers did eventually seize up and crash completely, and often there'd be some data lost. But, at least in my case, it was never much more than a skill point or two, or a few hundred experience.... twenty minutes of lost playtime. I'm sure some people had a worse experience than I did, that's the nature of random data loss, but I wasn't badly affected.

    When you consider the sheer scale of what they're doing.... they had TWO HUNDRED THOUDAND PEOPLE AT ONCE playing their game not long ago. The scale of that just boggles the imagination. If you assume 32kbits/second down, and 2kbits up (probably a bit skinny, but I'll try to err against Blizzard here), that's an aggregate total of 10,880,000,000 bits per second. Roughly 10 gigabits, or total saturation of an OC-192. Just the FIREWALLING on that kind of traffic is a HUGE project! Admittedly, they've broken that up into 3 or 4 datacenters, but doing firewalling and connection tracking on a mere 2.5gigabits is still pretty daunting. And that is completely ignoring the application servers, the load balancing, the inter-server communication, the databases..... just 1% of this project is a HUGE BIG DEAL.

    The fact that we were able to pile in there at that kind of speed and the game didn't seize up and die completely is a resounding, amazing success. I'm sure the Blizzard guys aren't feeling too great about how things went by now, but.... guys, you kicked ASS. You did somethiung that has never been done before, a level of complexity nobody else has ever reached, got loaded down with about four times as much traffic as you were expecting.... and STILL mostly succeeded.

    I have every faith that you'll work out the remaining kinks and bottlenecks.

    By the way, the last couple days, on Uther, have been quite good... I think I got one disconnect in two days, and there really hasn't been any lag. They may nearly have the problems fixed. I haven't been thinking about bugs, just about slaughtering beasties. :-)
    • You know, I would LOVE an article that explains the backend of the typical MMO server. Something tells me that most of the playerbase benefit from knowing whats going on in that server room.

      Life in the server room according to the ignorant forum whore.

      Admin 1: Hey, check this out! I was looking at the forums, and this lvl 60 Paladin, Pwnzj00, says "OMG UR SERVERZ SUXOR!!1!" Thats just not cool!

      Admin 2: *Stops playing quake* Really? I had no idea! What are we going to do about it?

      Admin 1: I d
  • There is one problem, and one problem only with World of Warcraft-- Blizzard isn't allowing character transfers across realms. All the problems we're talking about happen on something like 10% of the servers. When there are problems with my main character's server, I go play on a server that works. No one else seems willing to do this. To me, it's not a valid argument to demand your money back. There ARE servers up, just not the one with your primary character on it. If you're alliance, go do some hor

Slashdot Top Deals

Utility is when you have one telephone, luxury is when you have two, opulence is when you have three -- and paradise is when you have none. -- Doug Larson

Close