Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
PC Games (Games) Role Playing (Games) Entertainment Games

When Is A MMORPG Beta Not A Beta? 65

Posted by simoniker from the when-it's-ajar dept.
Thanks to Skotos for its editorial discussing the definition and previous use of the 'Beta phase' in relation to MMO titles. The writer argues, overall, that "The problems [in Beta] arise because of the differing agendas of the parties involved in bringing the games to and through the Beta process." He then posits that, "well before the full featureset is in place, the complexity surpasses the point where internal QA processes are adequate to cope", but "on the other hand, for the purposes of getting high-quality feedback that tells you what is wrong and where to look in order to fix it, Beta sucks." So, it's suggested, the end result is that "[massively multiplayer] Betas become exercises in community management, usually long before the team is ready to make the transition from developing a game to operating one. Meanwhile, an increasingly jaded marketplace is judging the Beta against the same standards they judge games at launch, or even years past their launch." What, if anything, can be done to ameliorate or fix these problems?
This discussion has been archived. No new comments can be posted.

When Is A MMORPG Beta Not A Beta? More

When Is A MMORPG Beta Not A Beta?

Comments Filter:
  • Companies don't have a network with a million nodes of users playing the game. So they call it beta and get it into the market.

    But at least they still call it beta. Microsoft releases windows and the real deal isn't the real deal until service pack 25 and 9000 hotfixes.
  • The real question most MMORPG gamers would ask is "When is a MMORPG not in beta?" Most games of this genre continue to add new features to the game as it progresses, thus keeping the playerbase. Everquest and its many expansions being a good example, not too many the countless things Sony has added between expansions. When are these games ever truly out of "beta". Though I do agree most beta tests dont accomplish as much as they would if they were for other types of software. Gamers tend to be gamers. Th
    • The real question most MMORPG gamers would ask is "When is a MMORPG not in beta?"

      That's an easy one. To quote a developer of one of the currently largest MMOGs I met last year, "When the money runs out, its time to launch!"
      • That's interesting, I heard almost exactly the same quote from a developer of an MMOG at a get-together in Virginia last year. Weird. Must be a truism in the industry... or you and I were both there, and your definition of 'largest' is a lot different from mine. ;)
  • Seriously shouldn't a piece software (be in MMORPG or OSes) be out of beta once it is on shelves collecting money from users.

    As the article says there is quite a lot of beta stages:

    * Development Beta

    * Playtesting Beta

    * Scaling Beta

    * Load Beta

    but these should not be an excuse for using users as guinea pigs (esp. paying users).

  • When is it not a beta? (Score:5, Funny)

    by AtariAmarok ( 451306 ) on Wednesday January 14, 2004 @03:54PM (#7977726)
    You know it's not a beta any more when you stop finding "SYNTAX ERROR: LINE 6778 INVALID CHARACTER AFTER ;" in treasure caves.

  • How About Incentives? (Score:5, Interesting)

    by illuminata ( 668963 ) on Wednesday January 14, 2004 @04:04PM (#7977834) Journal
    When a player successfully finds and reports a problem in the game, they could be awarded with extra stats, items, in-game money, etc. In fact, maybe real prizes like exclusive clothing, figurines, developer autographs, cash credit at an affiliate site, or just plain old cash would help out too.

    The prizes probably should be rated by severity of the issue that they brought to light. As they find more problems, they would get more prizes.

    It probably wouldn't hurt to give a special thanks page, possibly even a rankings page so that it could be turned into a true competition. The testers could receive a special title for when the game is officially released, too.

    Gamers tend to like free stuff that nobody else can get and respect within the game that they're playing. Any one of these ideas should be able to feed both needs.
    • they could be awarded with extra stats, items, in-game money, etc.

      Not for the final version, because then people who aren't in the beta will think they are at a disadvantage.

      The idea about a thanks page, and rankings page is probably a good one. Maybe prizes (tshirts, etc) for people finishing in the top 10 of bug finding?

      Of course, the game companies don't want this, they would be going through like 20,000 bug reports a day.
      • Not for the final version, because then people who aren't in the beta will think they are at a disadvantage.

        That's what I meant. The testing period should last a while so therefore those incentives would still mean something.

        But, maybe you could keep those in the final version with some restrictions. It would be nice to give the good beta testers (the top x) their own special area within the game after it launches where those incentives could still be used. As soon as you leave, those extras would be rem
    • A key problem with beta feedback is lack of quality in favor of quantity. More incentive may or may not improve the signal-to-noise ratio. You can get thousands of people telling you there is an item loss bug or a phantom dupe bug or a client crash bug, but until you get someone who can describe in detail exactly how to make it happen, its more of a rumor than a bug report. A major cultural problem is the beta players who just want a free chance to play the game before its released. When they find bugs
      • Well, a good development team would provide a clear outline of exactly how they want a bug reported. If a report didn't adhere to those guidelines close enough, they could just keep it in a rumor area for potential reference and to see if there are any rumors that seem to be similar. Any reports that are pretty bad could just be deleted with a message being sent out to the tester. I also had an idea for handing out demerits for duplicate reports in my reply to JavaLord, that way the tester would want to mak
        • It occurs to me that the Slashdot paradigm (are we still allowed to use that word in general conversation?) would be a great way to handle bug reports. Require some proper format for bug reports and have an open forum webpage where people can post and respond to them. Then have 'moderators' who give positive or negative ratings to the original posts and the responses.

          In time, the players who consistently write good bug reports, and those who provide useful feedback, such as alternate ways to reproduce
    • While that does work, it's mostly on the heads of designers, developers and QA to get a product out of beta.

      Anarchy Online had incentives like this through beta and still shipped in a state that left many people giving it the dubious title of one of the worst launched MMORPGs.
      • Exactly. Even the best beta testers can't save an incompetant development team. It's funny to watch game developers do whatever the gamers wish, no matter how detrimental. Sometimes the changes happen in beta, sometimes they happen after the game is released. Either way, the results make me chuckle.

        ...and the Canuck joke was great, dammit.

    • Re:How About Incentives? (Score:3, Interesting)

      by 1019 ( 262204 )
      Blizzard does this in a sense with their "Blizzard Friends" [blizzard.com] program where if you are selected for a Beta and you consistantly provide original and reproducable bugs in the game, you get added to the Friends list. The users on this list are automatically entered into the next Blizzard beta (whatever it will be..in this case the WoW Beta) but then they have to re-earn the right to be on the list for that one and so on and so on.

      I don't know if rewarding bug-hunters with stat/item upgrades would be in the bes
    • Shhh, the more bugs the users find, the more bugs the developers have to fix :D

      We don't want more work, now do we?

      lol

      Ignorance is bliss, as they say.

      - shazow

  • game betas aren't really betas (Score:3, Insightful)

    by MBraynard ( 653724 ) on Wednesday January 14, 2004 @04:04PM (#7977838) Journal
    Not the kind most people participate in. They are really advanced demos, meant to lock interest in for purchase among those who participate and to generate pre-launch buzz.

    And this is fine, especially for MMORPG games.

  • I have no idea! I play Shadowbane!

  • Possible Solution (Score:3, Interesting)

    by Josiwe ( 703514 ) on Wednesday January 14, 2004 @04:06PM (#7977861) Homepage
    A company like Square-Enix could very easily maintain a stable of beta testers across 5-10 games, rotating in fresh blood every time to keep the pool alive as certain testers grow older and have less time to beta test. After each beta cycle non-responsive or poor quality beta testers could be thanked and excused from service - they could even maintain two pools, one for actual testing and one for winnowing out efficient beta testers.

    Think of it like a job interview where the compensation is getting to play cutting edge games way ahead of the public as well as getting the developer's ear. And the company gets a small pool of very good testers (maybe 300) that they can count on to do good work and still be around for FFDQXXVI Online.

    • Re:Possible Solution (Score:4, Interesting)

      by MachDelta ( 704883 ) on Wednesday January 14, 2004 @04:57PM (#7978560)
      Atari has already started something like that. Its over at Betatests.net [betatests.net]. The basic idea is that you register an account, and then apply to various games to undergo beta testing. As you report bugs (though BetaTests.net) and generally be a good little tester, you gain 'points' and go up in 'rank'. The more points you have, and the higher rank you get, the more likely you are to be chosen for other tests by the developers surfing through your profile.

      Unfortunatly, its off to a bit of a rocky start. For one, there are a grand total of three things to test. And one isn't even a friggin game! There was a fourth game, Unreal 2: XMP (eXpanded MultiPlayer), but that test is now closed. The game/free addon was released publicly, and the existing crew of testers are being kept onboard to test future patches. Speaking of which, we're (yeah, i'm one) in the middle of testing a patch right now.
      In addition to the severe lack of games, the system doesn't even work properly. Bugs submitted through the website are generally ignored, or horribly delayed as they shuffle from Tester -> Publisher -> Developer and back. During the U2XMP test, we found it was *MUCH* faster, easier, and simpler to just use the forums to directly contact the devs. (And speaking of forums, BetaTests.net is supposed to provide forums for each test, but those were broken at the time of the XMP test too. So we wound up using a well respected fansite's [beyondunreal.com] forums instead). And to top it all off, due to the horribly inefficient reporting scheme, none of the developers seem to be taking (more like wasting) the time to actually give people 'points' to move up in rank.

      So all in all... good concept, but excecution is decidedly more difficult. Hopefully Atari can get things turned around in short order, but the system just isn't sustainable in its current state. Maybe once Atari figures out how to plow through this mess, other publishing groups will start similar projects. And who knows, maybe someone will eventually propose a universal database of testers?

      Anyways... just FYI, it does exist, albet tenuously

  • How to solve it? (Score:5, Insightful)

    by Edgewize ( 262271 ) on Wednesday January 14, 2004 @04:08PM (#7977881)
    It will be solved when you get publishers to stop making milestone dates based on fiscal reports and start paying attention to the development process.

    Rushing to beta is just as bad as rushing to release, but many developers are forced to do it because they are working on a timeline that has little room for internal review and repeated small-scale testing.

    Counter-example, Blizzard Entertainment and World of Warcraft: they have been testing internally for months, and the game is already stable and polished. When it hits beta, they will be able to manage it, because their "beta" will be more stable than some games that are on store shelves right now. That kind of development process is hard to come, and only a very few large developers have the clout to tell the distributor to STFU until the product is done.

    • Re:How to solve it? (Score:1, Informative)

      by Anonymous Coward
      Umm...the developer signs that contract with all those milestones on it. That means that they believe they can complete the game by date so-and-so. Yes, everybody admits that a certain amount of slippage is going to happen...but would it kill a developer to scale their development to the time period they have agreed to?

      Also, I've seen situations where small developers basically extort money out of a publisher by delaying their milestones once a project gets late in the dev cycle. The developer will then
      • "Also, I've seen situations where small developers basically extort money out of a publisher by delaying their milestones once a project gets late in the dev cycle."

        Yeah, cause you know the larger companies don't resort to dishonest practices as often :) I would imagine that word gets around between publishers about which developers are worth spending money on and which ones are pushing inferior products out of extended development cycles. The dev groups that are in it for the money instead of the games
    • That kind of development process is hard to come, and only a very few large developers have the clout to tell the distributor to STFU until the product is done.


      Hmm, who distributes Blizzard's games?
      • Hmm, who distributes Blizzard's games?

        Probably whoever their parent company is at the moment. id Software has had a similar relationship with Activision ever since id's developers made it well known that Activision was at fault for Quake 2 being released early. About every other project from Westwood (now part of EA) seemed to work out that way, too, almost in opposition of how their last game did (your last game did well, get this new one out now/your last game sucked because it was rushed, take your tim
        • I realize this, but do you know who their distributer is? Because I never heard of any parent company for Blizzard. Do they distribute their own games?
          • Last I knew, Blizzard was owned by Vivendi Universal. There were rumours recently that they might be sold off, but I hadn't heard that they actually were sold off. Sierra also used to be owned by Vivendi, and I'd imagine that if Vivendi had everything together, they'd distribute through the same company even though Blizzard wouldn't have Sierra's name on their box (since Blizzard isn't published by Sierra). They self-publish, more or less, but I don't know who does the distribution (in fact, I don't know wh
            • So therefore that's why Blizzard doesn't suffer from the same kind of pressure as other companies that depend on third party publishers/distributers. Blizzard has no one to answer to but themselves, hence the excellent qualtiy. :D

              - shazow
      • Hmm, who distributes Blizzard's games?

        Blizzard is owned by, and distributed by Vivendi Universal Games. The thing with Blizzard is, they have *alot* of star power. They've acquired that power because of the success, both financial and critical, of their games. Its because of that power that Blizzard is essentially in control of the developer/publisher relationship, Vivendi does what Blizzard wants them to do, not the other way around (as is the case with 99% of developer/publisher relationships).

  • Like Orwell said... (Score:1, Funny)

    by Anonymous Coward
    ...some betas are more beta then other betas.

  • When Is A MMORPG Beta Not A Beta? (Score:1, Funny)

    by Anonymous Coward
    When its ajar.
  • Games should have a preset 'Open Beta' for everyone who wants to download the game to participate in the test, because the most common crushing factor in a game at release is an overload on one factor in the network (usually a router somewhere down the line). This is often overlooked somewhat because the game developers are trying to perfect The Game (tm) and dont seriously factor in the network... which is surprising to see even after so many dismal launches of titles (AO, SB to name a couple).

    IMHO, a

  • How to "fix" the beta process (Score:3, Insightful)

    by Fortunato_NC ( 736786 ) <verlinh75.msn@com> on Wednesday January 14, 2004 @04:49PM (#7978478) Homepage Journal
    1. Don't think of beta testers as unpaid labor. Most beta testers are in it for a greedy reason - to get an early look at the next big thing. Spend more time in Alpha, working with PAID testers to refine gameplay and eliminate major bugs - use Beta testing as a final "shakedown", but realize that it's more usability testing than anything else.

    2. Keep it small. If you want to spend less time working on community problems and more time fixing code, don't open the beta process to everyone and their brother. Get people who are going to play and report bugs - if they don't, don't invite them to participate in the next beta.

    3. Develop comprehensive crash reporting, if possible. Tell the user what went wrong where, and give them the opportunity to send it to you via email or web. I can't imagine that MS got much good feedback from Windows beta testing when the only error people got was "General Protection Fault".

    4. Listen to feedback - nothing is more annoying than filing a bug report during beta and seeing the same bug in the shipping product. Have at least two people touch each bug report to avoid the "rubber stamp" syndrome.

    Of course, the fact of the matter is that a lot of MMORPG's use Beta testing as an opportunity to "hook em' early". They should offer a preview edition to do this, not get people who are more interested in playing games that squashing bugs disappointed with the quality of their product.
  • I didn't join until after it released and I paid out the wazoo for the privilege. I'd say Sony has a great beta setup - it never ends and they profit off it.

    When is a beta not a beta? Heck, it's better to ask when is a shipped product STILL a beta.
  • I've tried many times to get in on first round beta testing for MMOGs, and have given up. I have a lot of ideas for MMOGs, and I've had a lot of complaints about the current offerings. I can point to a long history of postings by me about these problems.

    Not to toot my own horn too much, but I think that it would be worthwhile for developers to seek out people who have posted a lot about other games for their initial betas - people who are highly opinionated and willing to put in actual time coming up with

  • Depends... (Score:2, Insightful)

    by Cosmik ( 730707 )
    Honestly, I think it depends on the individual players view as to when a MMORPG is no longer in beta - everyone is going to be looking for something different in a game. If the crafting system is fubar, then the MMOPRPG will still be in beta phase for the player who likes to craft; the PKer won't care at all (until he needs to buy phat lewt later in the game).

    For me, I see a few items on my checklist until I feel I'm no longer paying to play a beta:

    1. Billing system is operational: it stays up and worki
  • "Meanwhile, an increasingly jaded marketplace is judging the Beta against the same standards they judge games at launch, or even years past their launch."

    Well, if game companies actually gave a damn about what they claim are "finished products," where the released game didn't require as many patches as (if not more than) what a beta should be, we wouldn't be comparing the two as if there were no difference between the two classes (and really, are there?)

    Me? Jaded? Nah!

  • things that could be done (Score:4, Insightful)

    by XO ( 250276 ) <blade.eric@NoSPAM.gmail.com> on Wednesday January 14, 2004 @07:10PM (#7979972) Homepage Journal
    It should be made ABSOLUTELY CLEAR to the end users that they are TESTING THE ENVIRONMENT. Don't allow accounts for every person, their brother, sister, dog, and cat just because you want to load test your system. Load tests are easy to simulate.

    In addition to having very obvious ways (like a gigantic push button on the screen that says
    "REPORT BUGS BY CLICKING HERE" or some such) to report bugs, inconsistencies, and other things.. make sure that people are given credit where credit is due, for their testing work.. and send out WEEKLY surveys to ALL testers.. and they MUST be filled out and returned within a certain time frame, otherwise the user is cancelled from the beta program.

    These are ways to get quality testers. Even if they have no experience in software testing, you're at least going to get valuable input. Of course, people with software testing experience would probably be preferred, but also people with experience on other MM worlds would be great, too.. with the addition of several people who haven't the slightest clue (that's the best way to find bugs.. have people who don't know what they are SUPPOSED to do to get things to work right try things...)

    Make varied groups of testers travel together, presuming it's a traditional RPG... so you have people with test experience, people with MMrpg experience, and n00bs alike teaming.. sure, there's time for free for alls, but make time required to test certain things, as well.
  • I think a big problem with beta tests, and the preception of gamers, stems from these games ability to be internally patched. Because of autopatching, the devs feel they can release a beta way too early and then add in components as the beta progresses. That doesn't sound anything like the beta state in not-mmo games. That sounds more like an alpha state to me. Beta testing in offline games is usually about finding obscure bugs the devs would never have to time to pick up on their own and play-balancing

  • Beta should start when the game is usable enough to get customer input. This means that alot of content can be missing, the game can crash, there can be lots of nagging problems, but there is enough there for people to get the feel and respond.

    Most people who play games, especially MMORPG are not technically savvy, you want to get input from many different people on things like UI and general mechanics of playing because later on it is more difficult to fix those problems. Once those fundamentals are do

  • Betatests (Score:3, Insightful)

    by Lord_Dweomer ( 648696 ) on Wednesday January 14, 2004 @10:28PM (#7981774) Homepage
    Well, with MMORPGs, I'll be honest, when I betatest, I don't do it for them, I do it for me. Why? Well, frankly I am sick of all the MMORPGs that get released that could still be considered Betas. I'll be damned if I'm going to pay $60 for the box, and then $12/month to see if a game is worth playing. Yeah they offer 1st month free, but its a big hastle.

    Sometimes, if the beta was a good experience, I will purchase the game, as was the case with EQ and DAoC.

    Betas have become not something necessarily about testing, but rather giving a preview to all of the fans who have been eagerly watching the game progress from its conception.

    If a game company wants to capitalize on it, treat it as a promotional event. I'm in marketing/advertising, and from my PoV, thats exactly what these things are. For example, some games have given everybody tons of cash, lots of XP and skill points on the last day and let them go nuts. This is the kind of thing that can hook you customers. You see, if they get a taste for the high level stuff on the last day, they might be more willing to shell out the $60 for the box, and all the monthly fees necessary to get to the high level again.

    Perhaps the concept of the beta-test has evolved.

    • Must of bought The Sims Online. That game when it came out was only on public beta for 3 months before it was released. Us beta testers Told Maxis that it was in no way shape to be released and they should beta it longer. Well as soon as it came out lot of us beta testers exited stage left. I stayed in it for 1 year and let me say It is Still not ready. Content is missing that they said they were going to put in it since Beta and it is still buggy.
  • Ragnarock Online its got to be in beta for atleast a 3 or 4 years by now. dont know if it ever got out of beta thu stopped playing a long time ago. then in other massily long beta's look at Mame been over 3years and still not at 1.0 yet.
    • Dransik went into open beta for the first time in late 2000. It went back and forth between open and "closed beta" (closed beta=no new accounts allowed, only existing accounts. Open beta=free signup), and then went non-beta for a while in 2002. In late 2002, it went back to open beta with a new engine and some massive overhauling, which went to "pay-to-play beta" due to the developers going bankrupt last August. Then, TKO software bought Asylumsoft, and now it's back in open beta with a new name (Ashen Empi
  • Why not start a business specialiced in MMORPG Beta testing? I'm sure there is some kind of way this can work.

    Example strategy:
    Some helpdesk/support company can provide such a service. When they have a 'BETA Test project' they temporarly hire a couple of more agents to reduce the average load. Then they have all the agents play the game while they are not receiving phone call's and train them to correctly submit bug reports.

    Just a thought, but there must be a way to make profit out of this!
  • I think massively multiplayer online games should offer a trial basis - 1 free or inexpensive month. Paying $70 for a CD and $15 for the first month is a hefty investment for a buggy piece of crap.

    MMOG is a fledgeling genre that is dominated by relatively few products right now. Little competition equals little effort. It's a pretty safe bet any MMOG will be quite imperfect in the first 6 months of its operation. 2004 will introduce dozens and dozens of new games, each contesting for the same pool of
  • It's been happening for the last 10 years or so, but the words 'beta test' lost all meaning, judging from the replies here as well. There used to be a decent distinction between alpha test and beta test. Alpha means it isn't yet 100% complete, beta means it's basically finished and only needs to be tested before final release, with the possibility of bug tests. A beta test copy of software used to mean that it was the release candidate. Originally a beta test didn't mean the owner knows that it's full of b

Slashdot Top Deals

If all the world's economists were laid end to end, we wouldn't reach a conclusion. -- William Baumol

Close