Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Privacy Entertainment Games

Lies, Damned Lies, And Gaming Statistics 40

Thanks to the IGDA for pointing to a recent Chris Crawford-authored piece discussing the unreliability of statistics that reveal an ever-more diverse gaming public. The piece cites a recent Entertainment Software Association study and suggests that, because they don't reveal their methods or sources, "...the ESA results are unscientific. We can't place any confidence in them because we have no idea what they really mean. It doesn't matter how representative the sample is or how large it is or even if the researchers wore white lab coats." The ever-controversial Crawford goes on to argue that "Games people are smarting about the tawdry, unhealthy image that their industry suffers", and then offers a way out: "What's especially sad about this is that the solution to these problems is obvious: start making respectable games."
This discussion has been archived. No new comments can be posted.

Lies, Damned Lies, And Gaming Statistics

Comments Filter:
  • by M3wThr33 ( 310489 ) on Tuesday November 04, 2003 @05:12AM (#7384932) Homepage
    Numbers are critical to a game's success and in the past the industry has got the shaft in tracking sales for the most part. The industry-dominating NPD TRSTS don't even include Wal-Mart. Who's fault is that? No idea, but we really need to start modeling Japan's method. Each week they have it down pat.
  • To sum up the essay by Crawford, he complains that the published results [kensei-con.net] are not scientific. Unfortunately, an original document doesn't seem to be available from the ESA site [theesa.com], so all we have are a number of websites quoting/summarizing the results. This, of course, is not scientific, since important background information, e. g. definition of terms, is lacking (as Crawford notes correctly).

    To be reliable, any measurement or experiment must be repeatable

    So far, so good - but this, however, does not me
  • This is why you can't take surveys of your friends. If I did the same thing, I'ld get the reverse results, because almost everyone I know plays games more than they read books or go to the movies. Granted, maybe the people I know are wierd since most of them never see movies, but no wierder than his friends, many of which said 'never' to the video game question.

    He does make a good pointabout how the ESA survey is flawed, but then he tries to make his survey sound better, and then he falsely accuses the
  • by illumen ( 718958 ) on Tuesday November 04, 2003 @09:03AM (#7385495)
    Very interesting article :)

    """The games biz has nothing like this; we have hobbyists who slap together amateur efforts, and we have big-time people who develop the mainstream products. There's nothing in between, which is one reason why the games industry is so creatively stagnant."""

    I do not think that is so true. There are a few independant groups which make wildly popular games. eg Doom, snood, various other 'indies'. Over the last few years the indie game scene has really taken off. I think these people make games for a lot of people not into the mainstream games. For example many of them make games which will run on many computers, not just the latest ones.

    A game like snood, for example has gotten very wide public attention(In the multi millions). I've meet a number of non gamer types who have played it. The same with the doom/quake series of games. People who didn't normally play games at that time gave it a go because it was so interesting.

    Indie game developers can also afford to make games which do not sell lots of copies, because they are running more efficient businesses. For example instead of making 3-5% of each copy sold they can make 70-90% of each copy sold, and generally make games with much smaller budgets. They rarely do not sell as many copies as the publisher distributed.

    Take a look at the igf.com, dexterity.com/forums/. From there you will find lots of indie games, and evidence of such a scene. Check out gametunnel.com, indiegammer.com for reviews of indie games. There are really good original games in there, but also a number of crap games(like other indie, and publisher funded entertainment areas). Also check out ludumdare.com where there are occasionally 48 hour game making comps. Some very interesting ideas have come from there.

    Something like the Harvey Keitel effect*1 in pulp fiction is happening in other indie games. One occurance is where one/some of the makers of ufo XCOM are selling a game: Laser Squad Nemesis, http://www.lasersquadnemesis.com/.

    *1 Harvey Keitel effect - Having a big name giving more reputation, expertise and funds to an almost unknown group, drawing more attention to it. Like what happened in pulp fiction with Harvey Keitel. Because people knew he was on board, other people agreed to work with them.

    Have fun!
    holepit.com [holepit.com]
    • I do not think that is so true. There are a few independant groups which make wildly popular games. eg Doom, snood, various other 'indies'. Over the last few years the indie game scene has really taken off. I think these people make games for a lot of people not into the mainstream games.

      Yeah, but wildly popular in terms of those who have access to the 'Net, and know where to look. How many indie games do you see sold in mainstream stores (like the big retailers) or even specialty stores (like EB or Games

    • 'There are a few independant groups which make wildly popular games. eg Doom, snood, various other 'indies'. Over the last few years the indie game scene has really taken off.'

      Using Doom as an example isn't saying much. As for Snood, other than my brother who got the game via Spyware, I don't know anyone who played the game in my workplace, in my group of gaming friends, or any one in my family. Saying that a 'million' people downloaded it doesn't mean a 'million' different people played it.

      VERY few 'indie

  • Errr... what? (Score:3, Insightful)

    by DarkZero ( 516460 ) on Tuesday November 04, 2003 @09:29AM (#7385581)
    This person seems to be witnessing a completely different reality than myself and all those around me. We've either discovered travel between parallel dimensions or one of us is nuts.

    There's another consideration here: why are such polls so common in the games biz? It's not just a marketing slant; everybody in the games industry passionately believes this nonsense. Question this dogma and you'll be regaled with anecdotes of unlikely friends and relatives who are avid game players:

    Who passionately believes this? Not only are these statistics the biggest joke on every video game message board, chat room, or website I've ever visited, but if I remember correctly, even gaming magazines like EGM or TV shows like X Play have laughed them off. They're nothing but marketing propaganda and I have yet to see anyone take them to heart outside of a couple of ignorant, overzealous bloggers.

    What's especially sad about this is that the solution to these problems is obvious: start making respectable games. There is absolutely no technical or design reason why games have to be so tawdry. Sure, the primary market of adolescent males much prefers the violent rebellious stuff, and the industry can continue to make billions supplying this market. But the games industry needs to learn a lesson that the movie industry and the book industry learned long ago: you gotta push the noble fringe. Every year there are dozens of high-minded movies that get funding. Most lose money, but a few break even. Hollywood is willing to fund these efforts for three reasons:

    1. They support the image of cinema as an art form worthy of respect. Sure, Hollywood makes tawdry movies; moviemakers are certainly not above showing some skin to boost sales. But while there are plenty of individually sleazy movies, Hollywood's output taken as a whole is a healthily heterogeneous collection. For every Betty Bigtits there's a Mickey Mouse; Koyaanisqatsi played just down the road from Debbie Does Dallas.

    See, this is where we really get into that "parallel dimension" stuff. This man appears to be from a world where games like Grand Theft Auto make up the majority of the titles available. A world where there is no Ico, Mario Sunshine, Wind Waker, Jak & Daxter, Sly Cooper, Ratchet & Clank, Sonic Adventure (and soon Sonic Heroes), Smash Bros., Crash Band--- y'know, I'm gonna stop here, because with every platformer review I've read lately complaining about the "glut of platformers" in the vein of Ratchet & Clank and Wind Waker pushing out all of the other cutesy platformers that are worthy of being played, I could go on for several paragraphs.

    This author is apparently also from a world where the top ten list for American game sales isn't frequented by sports titles and platformers, but rather all of these "tawdry" games. In short, a world where BMX XXX sold more than two copies and returned copies of Enter the Matrix aren't cozying up next to E.T.: The Extraterrestrial for the Atari 2600 in a New Mexico landfill. A world where the game that GTA3 dethroned as the king of the North American sales charts for months on end was another tawdry, bloody game, rather than Gran Turismo 3 A-Spec. And, finally, a world where no one has tried out unusual, nonviolent games like Disaster Report (Zettai Zetsumei Toshi) and watched them fail miserably along with Ico, Rez, and Ikaruga, or occasionally watched them soar, like Gran Turismo 3 A-Spec, Dance Dance Revolution, or this little unknown game that's only released a couple of expansions... what was it called again? Oh yeah, "The Sims". I heard that game sold a couple of copies.

    I live in a world where there are not only plenty of respectable games, but at least as many of them as the bloody games. The only place where that differs is in the media, where nightly readers/viewers were once regaled with equally innacurate depictions of the comic book and movie industries, and where the only thing that will change that is the passage of time.
    • Ico, Ikaruga and Rez were not failures. They didn't become big hits, but their shipments sold out and just try to find used copies out there now. You'll have a hard time because the people that bought these games love these games and are hanging on to them.

      But I will agree with one thing. Ikaruga is fantastic.
  • For those who don't know who this guy is, he wrote some ground-breaking games way back when.
    Eastern Front for the Atari 800, Balance of Power for the early Macintosh, and others.

    He's also written extensively [erasmatazz.com] on game design.

Never say you know a man until you have divided an inheritance with him.