Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
The Media Entertainment Games

Defying Review Aggregation 53

Posted by Zonk from the don't-put-me-in-a-box dept.
Logiksan writes "With the growing number of review aggregation sites like GameRankings and RottenTomatoes, it's becoming increasingly harder for individual game critics to be heard. GameDAILY Biz took a stab at the issue at came up with 5 aggregation-defiant tactics designed to help make reviews relevant again. Among their list of ideas is to destroy the typical review grading curve. The article states, 'If, for instance, a publication could establish a 10 point scale in which reviews were based upon purchase value and average games scored only a 3 or a 4, the higher scores would certainly become far more important. The lower scores would give the publication instant credibility as 'discerning gamers' and would free up the top scores (5-10) to show a more full range of differentiation for the top-tier titles gamers care about most.'"
This discussion has been archived. No new comments can be posted.

Defying Review Aggregation More

Defying Review Aggregation

Comments Filter:
  • these guys ought to work together!

  • why? (Score:3, Insightful)

    by free space ( 13714 ) on Monday January 23, 2006 @06:20PM (#14544197)
    If I make good reviews, and they get aggregated with other reviews, why is this a bad thing that has to be 'defied'?
    Isn't the aggregation service something the readers persumably want?

    And why assume that the reader will only look at the score from your review? perhaps the reader is actually interested in the detailed information you provide and click on a link to your site from the aggregation site? And then the aggregation site will actually benfit the reviewer, not go against him.
    • I'll tell you why, and not because I'm a gamer fan, or a fan of anything specificly mentioned in the article, or your response. Aggregators creat clones. Period. I don't use RSS, I don't digg, or any of that crap. I read Slashdot, yeah, and there are times when it sucks, but that's hardly all I read during the course of my day, several sites, news channels, and newspapers, along with magazines or anything else I can get my hands on. Aggregators serve to add more bodies standing in the long line of morons al
      • I feel your pain. Humans in general don't like thinking, and prefer to replace it with the repetition/amplification of pre-canned thoughts. This has led to a lot of 'idea factories' ranging from political parties to aggregators, as you mentioned.

        But still. Stuff like slashdot and digg aren't all repeated crap, often you find genuinely interesting opinions from genuinely interesting people, and then it's worth wading through the other cliche's and buzzwords.

        Back to the topic, a game review agrregator would a
      • For all your rugged non-clone individualism and intellectual superiority, there are thousands and thousands of others exactly like you.

        Sad, but true.

        There is no "them." Only "us."
        • oh, believe me, there are very few like me. Come hang out sometime, you'll see what I mean. On that note, I'm sure there probably are some, but I'd much rather be like the very few like me, than the horde of others like everyone else.

    • Re:why? (Score:3, Interesting)

      by Daetrin ( 576516 )
      And why assume that the reader will only look at the score from your review? perhaps the reader is actually interested in the detailed information you provide and click on a link to your site from the aggregation site? And then the aggregation site will actually benfit the reviewer, not go against him.

      When i'm thinking about going to a somewhat iffy movie i'll check out RottenTomatoes and see what aggregate score it got. Then i'll look over the blurbs, keeping an eye out for any reviewers i tend to agree

      • I have to strongly agree, the aggregate score only tells me whether this game is worth researching, something with 20% average rating isn't even worth reading a review for (except maybe for any jokes the reviewer makes at the game's expense). If the score suggests that the game may be worth it I start reading individual reviews and form my final decision.

  • Aggregation not the problem (Score:5, Insightful)

    by yotto ( 590067 ) on Monday January 23, 2006 @06:27PM (#14544251) Homepage
    I don't read game reviews any more, but not because of access to aggregation sites. I don't read them any more because they are generally paid ads for the game or fanboys who just go on and on about how great it is that Joanna Dark's boobs are made from so many polygons.
    • Yeah, game reviews are quickly becoming irrelevant to quality. It's to the point where most gamers consider an aggregate score lower than 8/10 to be a "bad game." In their defense, it seems most reviews consider 5/10 to be their minimum score. So a 7/10 is only average, which (like other mass media) is pretty much crap.

      I'd be interested in a website with reviews that only focus on the bad parts of the game. Advertisers, demos, and promotions be damned: list off the major problems, along with the little an
    • That's why aggegation sites are great; just rate a few games you own, then compare your own scores to the scores given on an aggregation site and find out which, if any, of the individual review sites matches your preferences best.

      I guess that's why some publishers are against aggregation sites; it helps consumers pick the highest quality reviews, not the reviews backed by the biggest marketing budget.
  • Making the average score lower, but still out of 10 shouldn't make much of a difference. What if magazines/reviewers just decided to increase their maximum score without upping the aggregate average? ...but this volume goes to 11, see?
    • Another idea is to rate it out of 10... but don't allow for selecting the average score or intermediates. For example, rate on a scale of 1 to 10, but don't allow the selection of 7 or decimals. That way, when a truely average(7/10) game comes along, the reviewer must rate it and decide if it deserves that lowly 6 or that lofty 8. It's often a difficult decision.
      • I never quite got why "average" games deserve 7/10 (except for kickbacks from the industry...) Surely a bog-standard average game should get 5, a totally awful one should edge towards 0 and a magnificant one should edge towards 10 - currently the low numbers are under-used, and the ones at the top end (8 and 9 especially) provide insufficient room for reviewers to differentiate.
        Then again, perhaps trying to judge whether a game is worth 8 or 9 is pointless anyway and reviewers should stick to thumbs up, thu

  • From a former zine game editor.... (Score:5, Informative)

    by MBraynard ( 653724 ) on Monday January 23, 2006 @06:31PM (#14544290) Journal
    A lot of the smaller guys have trouble being heard - or ever heard of. The aggregator includes your zine's reviews with all the big boys. GameRankings is a big opportunity for free advertising.

    If you do a good job, readers may check out your review for a particular game (some people read all reviews) and decide to go directly to you from then on.

    Gamerankings is also great because they allow their readers to also rate the reviews themselves. So if you are a good writer, your reviews will stand above the mediocre ones when people look for reviews on a particular game.

    Maybe Gamedaily got tired of seeing so many of it's reviews receive one star or less?

    If I was really interested in starting a new zine from scratch, I would LOVE to get my reviews into the aggregator, try to accumulate high rankings for my reviews, and the traffic would increase and so would the willingness of devs to send me demo copies and put my quotes on their game boxes and I would also be able to get more advert revenue. It's a win-win-win all around.

  • Is this a problem? (Score:3, Interesting)

    by BertieBaggio ( 944287 ) <bob AT manics DOT eu> on Monday January 23, 2006 @06:51PM (#14544456) Homepage

    Now, I used to read reviews in magazines. I even had a subscription to Edge for a while, and used to like the longer reviews (the pieces on San Andreas from there and other mags come to mind). But now I tend to rely more on peer reviews - and by that I mean people I actually know. Combine that with selectively trying demos and I reckon I have a system that does pretty well (for me anyway).

    Of course, if I see a game discussed in depth on the world wide webby and with a great community, I will more than likely give it a shot. Notable games from this category are x/netrek; nethack; and HardWar, to name but 3.

    So who is suffering? Well, [formerly?] well-paid magazine or ezine professional game reviewers. And yes, that is a shame to some extent. I do enjoy reading the long reviews -- I appreciate when someone puts an effort in -- but I would rather spend the subscription money elsewhere, say on a new graphics card to take advantage of these great games I should be reading reviews of.

    So how do you recapture my (and everyone else's) attention? From TFA:

    Ditch The Scores - Makes the review stand on the merit of its content. Good idea - as long as the content is good.

    Focus on MegaReviews - Yup. El Goog prvoed that more is less; so write less reviews, but make them longer.

    Trumpet Your Own Credibility - Personality can be interesting, but a fine line lies between 'trumpeting credibility' and 'arrogant gits that I won't be rading again'.

    Aggregate Reviews on Your Own - Can't beat them, so join them? Might work.

    Crunch The Curve - If people are looking for a quick point score, they are looking for something that conforms to their expectations. Giving lower scores will ultimately damage credibility and turn off readers, even if your intentions are noble. Here's a better idea - lose the scores altogether. Give people a well-written indepth review. The ones that are looking for a point score won't read if you give it 4/10 as opposed to 7/10 anyway.

    my .02

    • If you ask me the whole premise of the article is the 3rd point, "Trumpet your Credibility". It seems to be based on the assumption that the reviewers opinion is much more important than any of the other reviews being aggregated.

      Personally, I would prefer to get an overall picture of whether a group of reviewers liked a product rather than the individual opinion of a reviewer who may or may not be carried away with his own sense of hubris. I may not hold the same opinion on the subject of the review as him,
    • The ones that are looking for a point score won't read if you give it 4/10 as opposed to 7/10 anyway.

      Not true. I like to read the highest and lowest scores for anything I'm going to buy because these reviews between them will give me the clearest picture of the pros and cons.

      I recommend this approach, by the way. Works well if you don't have time to read every review of everything.
    • "Ditch The Scores" - Baby, bathwater.

      "Focus on MegaReviews" - If I read two or three reviews per movie I actually go see, and the review takes me half as long to read as I would have wasted just going to see the movie, then I may as well just go see the movie and skip the reviews.

      "Trumpet Your Own Credibility" - Because we all know that nothing impresses us more than hearing someone brag...

      "Aggregate Reviews on Your Own" - Hey, no fair, you guys suck so we'll do the same thing!

      "Crunch The Curve" - a

  • Finally! (Score:4, Interesting)

    by Tom ( 822 ) on Monday January 23, 2006 @06:59PM (#14544536) Homepage Journal
    Been about time. When I stopped regularily reading gaming mags 5 or so years ago, 70% (or 7/10 or whatever the scale was) was a pretty mediocre result, and it's only got worse. The same magazines that once celebrated the first ever game to score over 90%, ever - are now giving out 90% every month or so.

    It's all marketing pressure. 100% does not mean "perfect" anymore. Problems that would've dumped the score 5, 10 points are shrugged off by many testers as "will certainly be fixed by the next patch" or "but it's still better than elsewhere".

    There is one online review page that still writes very criticial and sometimes harsh reviews, where the stuff that rates 94% in your average mag (which only by coincidence has a two-page ad by the same company that month...) - well, that overhyped crap gets its 47% or whatever it's really worth once you remove the "big names" and the photoshopped screenshots.

    I just wish for the life of me I could remember the URL. I lost my bookmarks once, and that was the only game review site worth having a link.

    I also remember what some of those tester dudes said when flat out confronted with the fact that they only ever seem to review 70%+ games. They said "we don't want to waste precious magazine space with the mediocre games".
    Sounds believeable. Except that the rations are still way overblown. I like "The Movies", for example, but it's not a 9/10 game. Civ4 - great game, but 9.7/10? If you can only imagine 3% missing to absolute perfection then you have damn little imagination. And so on, and so forth.

    Really, an honest rating should either allow > 100%, or say "90% if you do everything in the best way that I can imagine. Points above that only if you found better ways to do it, and because you had a few years to work on it that's not an unrealistic expectation".

    • Actually, it seems the aggregate sites are worth something. Looking for what side scored a few games very low I'd also consider failures but which were hyped anyways, I stumbled upon:

      http://pc.gamedaily.com/.

    • Re:Finally! (Score:3, Interesting)

      by Red Flayer ( 890720 )
      "Really, an honest rating should either allow > 100%, or say "90% if you do everything in the best way that I can imagine. Points above that only if you found better ways to do it, and because you had a few years to work on it that's not an unrealistic expectation"."

      This is where game reviewers could learn something from an established idustry that faced the same problem decades ago -- fine wines.

      Robert Parker and other wine critics theoretically can assign values up to 100. But a 98 is a once-in-a

  • why the need for a numerical rating? (Score:3, Insightful)

    by usrusr ( 654450 ) on Monday January 23, 2006 @07:01PM (#14544555) Homepage Journal
    while the suggestion to increase the actually used "dynamic range" of the scoring system used is certainly a step into the right direction, why does he stick to the an overall numerical rating system at all?

    i could imagine very good reviews that would group the usual categories into pairs of that are contradicting each other or at least are quite opposite to each other, like "this game focuses on replayability" vs "this game focuses on an intense story", "focuses on technical aspects" vs "focuses on other qualities", or just distribute a _fixed_ number of points on the various categories, to describe the game, give it a rough position in an n-dimensional matrix, not rate it.

    a serious review should never pretend to be able give ratings with more precision than something along the lines of "you will love it"/"you will like it"/"you might enjoy it"/"pain" (all assuming you generally enjoy the genre).

    "game x is 3.5% better than game y" is pure bullshit and a sure sign of a review that is actually nothing more than comparing technical specs and skipping through the game with cheat codes to provide some screenshots.

    and yeah, pissing off game publishers with honest ratings is a really bad idea in a business that mostly depends on hyping up "exclusive" previews to lure customers.

    the threat of losing "early screenshot" benefits is probably an even bigger threat than the whole dependency on ad money from publishers, not only because it looks less like bribery but also because customers lost due to lack of exciting "next gen" cover stories will affect the ad-income from all companies, not only the one in question.
    • to conclude: reviews can only really be reliable if they are either from a noncommercial source (but beware of fanboyism) or if they are from a more general source that does not primarily focus on games.

      (ps: how many weeks would it take to give a meaningful judgement about a game like civ4 for example? imagine the sales of a game mag that would make a civ4 cover story 4 weeks after the release of the game...)
      • and yeah, pissing off game publishers with honest ratings is a really bad idea in a business that mostly depends on hyping up "exclusive" previews to lure customers.

        Giving a crappy game high ratings to appease the publisher hurts your reputation among gamers. That means less visitors, ad clicks, presence, and relevancy. Game reviewers exist to provide a service to gamers interested in purchasing games, not publishers wishing to sell them.

        reviews can only really be reliable if they are either from a non
        • Giving a crappy game high ratings to appease the publisher hurts your reputation among gamers. That means less visitors, ad clicks, presence, and relevancy. Game reviewers exist to provide a service to gamers interested in purchasing games, not publishers wishing to sell them.

          but do you see that happen? of course there are games that get lower ratings than others, but those are games where the publisher probably would not expect a score reminiscent of hl2 themselves. as long as you "vote with the crowd" (yo
    • why does he stick to the an overall numerical rating system at all?

      Because game magazines aren't there to educate their readers. Their purpose is to sort game producers by advertisement money spent and then arrange their games in a simple list of "most worth buying" to "tell them to buy more ads next time".
  • Who is the author's target audience? The sort of people who are going to make a purchasing decision based solely on the weighted average of an aggregator without reading any of the linked reviews? Obviously not. The people who use an aggregator but do read a cross-section of the linked reviews to get context and multiple points of view? Not them, either.

    So who's left? People who don't use an aggregator in the first place. And what's he imploring them to do? Not use an aggregator. Seems kind of poi

    • Look at Gamespot, the page is flooded with 8 layers of advertisements. Gamerankings has ads, but it is still orderly and not flash flooded. Sometimes the best comments are those from ebworld and amazon where the consumer happens to discover something negative 6 weeks into owning a game. Now that is helpful.

  • Game Reviews: Obsolete? (Score:3, Interesting)

    by Sage Gaspar ( 688563 ) on Monday January 23, 2006 @07:34PM (#14544815)
    I think reviews, themselves, are becoming outmoded. I can go to a message board and get opinions from hundreds of other players that I know share similar interests with me. They know what I'm going to care about in a video game more than some random shlomo.

    Video game magazines tend to be targeted at a different audience than me, and I'm not entirely sure what that audience is: maybe younger gamers, or those who aren't quite as involved. You get the opinions of one or two people, max, and possibly short scores from a handful of others that I don't know from Fatal1ty (Adam for the less hardk0re :P).

    Finally, before I plunk down money on a game I try the demo. Even a lot of console games now have computer versions with demos. They might lag behind, but I usually stay a year or two behind the console trend because the pricing on their games are ridiculous (some computer games are guilty of this as well, but they tend to be more reasonable).
  • Why not just allow anyone to write reviews and give game scores? Then the readers moderate the reviews, and can set threshholds of what sort of modded reviews they want to see.

    What's that? Trying this good idea from the rest of the world would put professional game reviewers out of business? Well then they should change their business model, perhaps move into those game-reviewing websites which'll likely have pretty decent ad revenues, or possibly a charge for membership.

    Has this been done elsewhere or s
    • I want to read a review from a guy who's reviewed 200 games before and who can be convincingly objective to me. I don't want to see a bunch of 10/10 reviews from fanboys who think that the game is completely misunderstood by the Mass Media, and I don't want to see a bunch of 1/10 reviews from people who will condemn a game they hate based on a Penny Arcade strip that made fun of it.

      Take a look at amazon.com, gamefaqs.com, and ebgames.com to see how awful the result is when you let everyone review a game re
      • Funny, because at least Amazon should be able to know if you've bought the game from them.

        So a decent review site has to have somebody who can be the Aloof Objective Guy. Funny, but on Slashdot these are called the editors and everyone hates them. So why not just have both things? A special section or moderation rating (ie: a separate scale entirely) would be given to Real Objective Reviewers and you'd go to another section if you wanted the People's Reviews.
        • Gamespot does that. They provide an editorial rating and then let readers rate the game separately. The "objective guy" rating can differ significantly from the "people's review" average. Here's one game [gamespot.com] that got a mediocre review (5.0) for being "too self-referential" about the anime series on which it's based; not surprisingly, the People rated it significantly higher (7.6). Gamespot also provides links to other review sites, whose reviews average out to be between the two (6.6).

          Now because this is Sl
          • I like the Gamespot system, as well.

            I also don't usually read the reviews on websites, which apparently angers the people who write them. Waaah. It's easier to read Game Informer while I'm taking care of umm...non-gaming "business."

  • Bizarre. (Score:2, Interesting)

    by Retroneous ( 879615 )
    My site relaunches on Friday 27th and we decided to "drop the scores" a couple of months ago.

    They're pointless in the current climate. Personally, I decided to take this route because I was sick of seeing referral traffic from forums where folks had come along to the site, read the score (not the review text) and then promptly headed back to their favourite forum to trash us.

    "Why did he give that 9/10? It sucks! Its not as good as !" etc., etc.

    The alternative? Just give them the review. Then they don'
  • Gamerankings already changes a review's score to work on a 0-100 scale. If they know that an "average" score for a site's reviews is a 3-4 and that compares to a "7" on other sites, they'll just mark them down as a 7.

    Besides, the whole thing smells of sour grapes to begin with.
  • The article states, 'If, for instance, a publication could establish a 10 point scale in which reviews were based upon purchase value and average games scored only a 3 or a 4, the higher scores would certainly become far more important. The lower scores would give the publication instant credibility as 'discerning gamers' and would free up the top scores (5-10) to show a more full range of differentiation for the top-tier titles gamers care about most.'"

    This is easily accomplished doing something instead o

  • http://www.gamefaqs.com/ [gamefaqs.com] and http://www.gametrailers.com/ [gametrailers.com].

    The point for me is that the score doesn't matter one little bit. What is important is *why* they gave it that score and/or why they didn't.
    For example the reviewer might mark a game down because it was too hard. For me that would be a plus. They might say it is fun but too short, which also might appeal to me at certain times.
    What is also important for me is to see some actual gameplay. I can often pick up more about a game from a 2 minute vi
  • Write reviews which are more than just a description of the game with some suitable adjectives and a score attached. "Amazing level design, great graphics, 10/10" and "Horrible level design, losy graphics, 1/10" aren't really interesting to read.
  • Sounds to me like the games industry is bitching because they make it that less effective and more expensive to bribe up a handful of review scores using advertising revenue or whatever.

  • Edge magazine regularly give out 5's for average games and only give out a ten once every year or two. Perhaps it's not on the reviewers radar because it's a UK publication. http://www.edge-online.co.uk/archives/edge/index.p hp [edge-online.co.uk]
  • If the people writing these reviews possessed any modicum of ability they wouldn't need to use numerical scores. A well written review should tell you everything you need to know about the game that is required to make a purchasing decision. All too often the content of the review and the score that accompanies it have nothing in common; the scores seem to be nothing but an over inflated fop to advertisers.

    Take one of the few quality gaming magazines that still exists, say Edge, and compare it to an inter

  • I was thinking about writing game (and maybe other) reviews for my newly created site, and one of my ideas was to either ditch the score system completely, or make it very simple, almost binary. Assigning 0/1 didn't seem fair though, why should a game which barely redeems itself get the same score as a GOTY-class title? Right now I think giving out average/recommended/highly recommended/second coming of Christ ratings would give a clear idea of what my feelings were, without getting into the whole "give thi
  • /donning Flameretardant

    I personally like a video review and I get most of mine from X-Play on G4. I know they are whores but they are honest whores. Nothing captures the essence of ultimate spiderman like the world's rudest Stan Lee impersonator. More importantly I can see the gameplay and Sessler is really fond of demonstrating why a game sucks showing you the terrible camera or ai or whatever. Plus Morgan is pleasing to look at if not listen to.

    I also get a few mags like gamepro and I read every revie

  • I work for a progressive review site called http://www.thegamechair.com/ [thegamechair.com] and would like to mention a couple points. First, we would love to be included on some of the major content aggregators. The major aggregator's sites function something like a Google search for reviews with the handy addition of a mathematical formula for determining the average score. I feel like they drive out far more traffic out to individual sites than they steal from them. I know for a fact that I regularly read 5-10 reviews

Slashdot Top Deals

I put up my thumb... and it blotted out the planet Earth. -- Neil Armstrong

Close