Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
×
The Internet Entertainment Games

Investigating Bias In Videogame Review Sites 64

Posted by simoniker from the biased-towards-printing-this dept.
jvm writes "We've all read comments that some videogame sites are allegedly biased for or against some games, consoles, or companies. So, Curmudgeon Gamer has investigated whether bias can be seen in the review scores over several games on each console. The review sites in question are GameSpot, GameSpy, and IGN, each of whom are compared to the game review averages on GameRankings. Additionally, a selection of review scores for crossplatform games are examined. While solid conclusions are difficult to draw and improvements can admittedly be made, perhaps people will find these results interesting to examine and discuss."
This discussion has been archived. No new comments can be posted.

Investigating Bias In Videogame Review Sites More

Investigating Bias In Videogame Review Sites

Comments Filter:

  • Running a game review site myself... (Score:5, Insightful)

    by tvalley000 ( 410933 ) on Tuesday November 04, 2003 @06:49PM (#7391341) Homepage
    I have to say, from the game reviewer's standpoint, that it's hard to give a game a bad rating.

    1) You have advertisers. Typically, these people keep your site alive. They're expecting you to review their game.

    2) You like games.

    3) You get tons of swag, press passes, the royal treatment at trade shows, and a ridiculous amount of geek cred.

    4) Did I mention that you like games?

    5) You get to mention Sanya Thomas' butt in casual conversation. (see?)

    6) It's a freaking game. What's not to like about a game? (well, unless it's Final Fantasy XI -- which I refuse to admit is actually a game)

    Mystery@Warcry.com

    • Re:Running a game review site myself... (Score:1, Insightful)

      by Anonymous Coward
      not to mention that reviews of media of this nature at best are highly subjective.
    • You forgot the Editor... the position where the one with the Big Thumb can thump a candidate review back to draft status for adjustments. The EIC is responsible overall for a publication, and can be 'coached' to tell his team how to play.

    • if you can't do a bad review, please quit (Score:5, Insightful)

      by DrSkwid ( 118965 ) on Tuesday November 04, 2003 @07:58PM (#7391956) Journal
      because you aren't doing anyone a favour.

      If I buy a $50 game based on your review then I can never trust your opinion again.

      Regular readers should notice that you never give bad reviews.

      If you can't be honest then you are *just* advertising not "relying on advertising".

      • because you aren't doing anyone a favour.
        If I buy a $50 game based on your review then I can never trust your opinion again.

        Hey, I was making a point about the way advertisers and PR companies promote games, and the influences they apply to otherwise inexperienced volunteer "journalists". I wasn't stating that I, personally, only give games good reviews.

        Regular readers should notice that you never give bad reviews.

        Regular readers would know to make the distinction I made, above.

        If you can't be hone
        • Sorry that you felt unjustifiably attacked.

          Your opening sentence sounded more like personal experience than observation.

          "I have to say, from the game reviewer's standpoint, that it's hard to give a game a bad rating."

          I sincerely hope that you do have an honest relationship with your readers.

      • because you aren't doing anyone a favour.

        If I buy a $50 game based on your review then I can never trust your opinion again.

        Regular readers should notice that you never give bad reviews.

        If you can't be honest then you are *just* advertising not "relying on advertising".

        Whilst this is the ethical stance, and I am in agreement with your opinion, it would appear that magazines and websites have nothing to lose by printing the reviews their advertisers want. If there was an impact on sales or page views

      • Exactly DRSkwid. Why the hell do I want to read ANYTHING from a review site where reviewers have a hard time giving a game a bad score?

        You (the writer of the parent post, not Skwid.) must write for IGN. The whole reason I started writing reviews, was not for the swag, it wasn't for the free games, it was because I was a college student, and I decided that SOMEONE was going to tell people when a game really sucked so they didn't spend their $50 on a piece of crap.

        PC Gamer wasn't doing it, Gamespot wasn't
    • I write reviews for a site as well. As well as do some news when time permits, and editorials.

      However, we don't get a whole lot of PR treatment from companies. We all either buy or rent the games we review, ourselves, over 9 out of 10 times (rarely do we get a review copy sent to us, but it does happen every now and then). We don't get wined and dined by PR comapnies. And, most importantly, we have very little advertising on the site, except from our association with Amazon.com and Ebay (and even thes

      • Not all gaming sites get the treatment that IGN, GameSpot, or GameSpy get, you know.

        Of course not. Otherwise, you wouldn't be able to list them off like that.

        The subject in question, and the thread to which you were responding, was questioning the ethical quandry faced by typically volunteer reviewers that are submitting otherwise unprofessional works to outlets where advertisers pull the strings through the use of gifts that are heartily appreciated. The gaming site that you described doesn't fall int
        • Last I remember, most of GameSpot's editors are actually employed by GameSpot (C|Net), and thus are not part of the "...typically volunteer reviewers that are submitting otherwise unprofessional works to outlets..." that you mention.

          I'm not sure about the employment status of IGN's or GameSpy's editors.

          As for other sites which rely on advertising to pay their bills and use volunteers, well, I can't say what they do or don't do. We don't rely on our 2 advertisers to pay our bills (in fact, I paid our se

  • The question is (Score:4, Funny)

    by octover ( 22078 ) on Tuesday November 04, 2003 @06:51PM (#7391352) Homepage
    The question is there no bias in this investigation of bias in video game sites?
  • Unless you were just born with the knowledge you'd have in 35 years but had no opinions, all reviews are gonna be a little biased, i mean cmon, reviewing games arent a mathematical thing, its giving your opinion.

  • Missing the point (Score:2, Insightful)

    by Shadarr ( 11622 )
    It's mildly interesting to see that there may be an anti-GameCube bias in some reviews. However, what I would much rather see is the same methodology applied to other factors, like who the publisher is and who is advertising with a specific site. I'd never heard anything about specific-console bias, but I've darn well heard an earful about EA getting a free-ride on steaming piles like Earth & Beyond. I'd like to see whether the numbers back that up.
    • I don't see an anti-gamecube bias when I read these sites. I think in most cases a gamecube port is worse than the other consoles. #1 because of the controller (and not as many buttons as other consoles). This leads to more complicated control schemes on a lot of games. Also the layout is bad IMO. #2 The media. I know IGN gave SSX Tricky a lower score on Cube because it did not have as much FMV (just 1 example). #3 Lack of any online component. Nintendo only has 1 game online. And only a handful can

  • Meta review sites (Score:1, Interesting)

    by Anonymous Coward
    This article just highlights to me the benefit of meta review sites such as Game Rankings and Meta Critic. User review scores also help counter any bias in reviews. What I'd like to see is a comparison of official magazine review scores compared to the average. OXM would probably turn out to be far higher than the average - especially for Microsoft published games.

    All in all, the Internet is a powerful equaliser in these matters.

    • Re:Meta review sites (Score:4, Interesting)

      by MMaestro ( 585010 ) on Tuesday November 04, 2003 @10:18PM (#7392920)
      True user review scores can help counter bias in reviews, however keep in mind that readers themselves can become bias. Especially in games that have long outstanding reputations or big brand name backup. For example :

      On Gamespot, they list GTA3 for the PC at a score of 9.3 (not bad since the PS2 version got 9.6). But lo and behold, the reader's score(s) drop the score by .3 points for the PS2 version and an even harsher .8 points. If you're concerned about small bias (a couple people picking 1's) : 19578 readers voted on the PS2 version and 2454 readers voted on the PC version. Hard to sway that kind of majority..

      However, vice versa this also happens :
      Gamespot (again) lists GTA2 for the PC at a fairly low 6.8 (OUCH compared to GTA3's score) and the PS1 version at 6.9. But insert reader reviews and GTA2 for the PC shoots up a whole 1.4 points and the PS1 version almost breaks acceptable levels with an insane 1.9 points! Nearly 2.0 whole points, and just .2 away from the much coveted 9.0 spot.

      Keeping this in mind, its VERY easy to see how jaded or how harshly the public can "vote" when it comes to their favorite games. Throw a "reader review" system on Slashdot for programs between Windows, Linux, and Macs; and Windows will be lucky to get a score of 1.2 .

      P.S. If you want to check the scores, use the GameFinder on the left sidebar on Gamespot.

  • Gaming still lacks professional reviewing (Score:4, Insightful)

    by superultra ( 670002 ) on Tuesday November 04, 2003 @09:05PM (#7392477) Homepage
    Curmudgeon Gamer's examination is certainly an interesting one, but what it complete neglected was the trend of increasing stupidity in gaming reviews.

    Perhaps the slow but steady rise of ludology [ludology.org] means that we may yet see a rise in the quality of game reviews, but by and large the current level of intelligence in most game reviews is as low as the number of female companions most of the game reviewers have (and I mean that collectively). Reviews inevitably follow the "Background, Cool Trivia About Game, Graphics, Controls, Bad, Good" template. The size of the font of the scores at the of end review keeps getting bigger, as does the size of the screenshots. Basically, we're left with an industry wide standard of two paragraph GamePro-esque reviews that generally boil down to, "It rocks!" or "It sucks!" EGM's recent redesign is a fine example of this. There is little in the way of anaylsis, or the game's relation to the industry at large, or (heaven forbid) society itself. I think there are sites on the net that are the exception, such as GameCritics.com, and ironically, Penny-Arcade. Likewise, I think either Computer Games or CGW (I always get them mixed up) is making a conscious move in the other direction. Kudos to them, and shame on the rest of the reviewers. I suppose that online is, strangely enough, more "textual" than the magazines due to the lack of space confinement. But the actual content is about as low.

    What I'd like to see is a complete abolishment of scores all together in a magazine or website. It would mean, yes, you'd actually have to read the review. That's not to say there's no place for the blurb-y Gamepro-style of reviews with a big thumbs up or thumbs down at the end of the review. There obviously is, as it's quite handy for a quick reassurance when you're in the line at EB almost ready to buy the game.

    But there needs to be more peer reviewing, more intelligent discoursing, and more analysis ala ludology in the gaming industry. Not everything is the next best thing since, well, the last game the person reviewed. But some things most definitely are, and it's these that need extra attention. Not, contrary to popular belief, more screenshots.

    • An addendum (Score:3, Insightful)

      by superultra ( 670002 )
      I forgot to mention Play magazine. I don't particularly always agree with their reviews; I think that Gunvalkyrie is difficult not because of content but because of poor control design, and so on and so forth. I feel that at times Play tries too hard to be "hardcore." But Play magazine provides a niche, instead of trying to be *THE* magazine for gamers as [insert game magazine name here] is so fond of doing. You have to admire that.

      Likewise, I thought the reviews in the defunct NextGen were absolutel

  • my two cents (Score:1, Informative)

    by Anonymous Coward
    my only problem with the article is that it is based on USER REVIEWS (the term he keeps using), and not the SITE REVIEWS, when the article was stated to be about whether or the SITES are biased and not if their READERS were biased. This also doesn't take into account the people that have no lives and will sit their and submit bad Xbox/PS2/GC game reviews on gamerankings without even owning the system, just to show their fanboyism.

    Sorry I'm posting as Anonymous Coward, I just don't feel like setting up an
    • my only problem with the article is that it is based on USER REVIEWS (the term he keeps using), and not the SITE REVIEWS, when the article was stated to be about whether or the SITES are biased and not if their READERS were biased

      The article uses 2 sets of user reviews and 3 sets of site reviews and clearly distinguishes between the 2, as well as having some discussion of the differences between site and user reviews. It uses the GameRankings compilation and averaging of reviews as the basis against whic

  • so, really the only conclusion... (Score:3, Insightful)

    by fireduck ( 197000 ) on Tuesday November 04, 2003 @10:47PM (#7393084)
    that can be drawn is that game players are harsher than the professional critics. no duh. visit any official or psuedo official forum for a game, and chances are you'll encounter a rather vocal group complaining about problem X. never mind that 95% of the players don't encounter problem X, but the 5% that do, really want to shout about it. I imagine the same thing would happen with reviews. One is much more likely to bitch about a game that was less than satisfying than praise one that was enjoyable (as you get satisfaction from the game in the latter case, and are satisfied by bitching about it in the former). So the scores are biased in the downward direction. that isn't too surprising.

  • I always love reading articles with statistics. I'm glad that this article was written.

    The first thing to keep in mind is that the writer predominately chose games which generally had received high review scores. So, in essence this article was really about whether there is a review bias among highly-rated games. I think this point is important because it means that each site is generally debating exactly how "very good" each of these games are.

    Which brings us to looking at the rating systems of these
  • Good stuff. My one concern is that by choosing only top rated games, he's built-in a flattening effect that will reduce any appearance of bias. To be a "top" game, by necessity the numbers are going to average high, which means less variation.

    An "average" rate game is more likely to have a wider variety of scores. Examining games with average ratings (around 60-80%) would probably yield a better picture of any bias (if existant).
  • From the article:

    While I'm at a loss to explain just why GameRankings users score the way they do, it is interesting that their opinions of games are consistently lower than the average of all game review sites recorded on GameRankings. Furthermore, the Xbox is by no means alone in this respect; GameRankings users are similarly critical of the top twenty games on the PS2 and GameCube.

    It should be noted that the user scores from GameRankings [gamerankings.com] are going to be well below the average for all platforms. As

Slashdot Top Deals

Whoever dies with the most toys wins.

Close