Defying Review Aggregation 53
Logiksan writes "With the growing number of review aggregation sites like GameRankings and RottenTomatoes, it's becoming increasingly harder for individual game critics to be heard. GameDAILY Biz took a stab at the issue at came up with 5 aggregation-defiant tactics designed to help make reviews relevant again. Among their list of ideas is to destroy the typical review grading curve. The article states, 'If, for instance, a publication could establish a 10 point scale in which reviews were based upon purchase value and average games scored only a 3 or a 4, the higher scores would certainly become far more important. The lower scores would give the publication instant credibility as 'discerning gamers' and would free up the top scores (5-10) to show a more full range of differentiation for the top-tier titles gamers care about most.'"
Is this a problem? (Score:3, Interesting)
Now, I used to read reviews in magazines. I even had a subscription to Edge for a while, and used to like the longer reviews (the pieces on San Andreas from there and other mags come to mind). But now I tend to rely more on peer reviews - and by that I mean people I actually know. Combine that with selectively trying demos and I reckon I have a system that does pretty well (for me anyway).
Of course, if I see a game discussed in depth on the world wide webby and with a great community, I will more than likely give it a shot. Notable games from this category are x/netrek; nethack; and HardWar, to name but 3.
So who is suffering? Well, [formerly?] well-paid magazine or ezine professional game reviewers. And yes, that is a shame to some extent. I do enjoy reading the long reviews -- I appreciate when someone puts an effort in -- but I would rather spend the subscription money elsewhere, say on a new graphics card to take advantage of these great games I should be reading reviews of.
So how do you recapture my (and everyone else's) attention? From TFA:
Ditch The Scores - Makes the review stand on the merit of its content. Good idea - as long as the content is good.
Focus on MegaReviews - Yup. El Goog prvoed that more is less; so write less reviews, but make them longer.
Trumpet Your Own Credibility - Personality can be interesting, but a fine line lies between 'trumpeting credibility' and 'arrogant gits that I won't be rading again'.
Aggregate Reviews on Your Own - Can't beat them, so join them? Might work.
Crunch The Curve - If people are looking for a quick point score, they are looking for something that conforms to their expectations. Giving lower scores will ultimately damage credibility and turn off readers, even if your intentions are noble. Here's a better idea - lose the scores altogether. Give people a well-written indepth review. The ones that are looking for a point score won't read if you give it 4/10 as opposed to 7/10 anyway.
my .02
Finally! (Score:4, Interesting)
It's all marketing pressure. 100% does not mean "perfect" anymore. Problems that would've dumped the score 5, 10 points are shrugged off by many testers as "will certainly be fixed by the next patch" or "but it's still better than elsewhere".
There is one online review page that still writes very criticial and sometimes harsh reviews, where the stuff that rates 94% in your average mag (which only by coincidence has a two-page ad by the same company that month...) - well, that overhyped crap gets its 47% or whatever it's really worth once you remove the "big names" and the photoshopped screenshots.
I just wish for the life of me I could remember the URL. I lost my bookmarks once, and that was the only game review site worth having a link.
I also remember what some of those tester dudes said when flat out confronted with the fact that they only ever seem to review 70%+ games. They said "we don't want to waste precious magazine space with the mediocre games".
Sounds believeable. Except that the rations are still way overblown. I like "The Movies", for example, but it's not a 9/10 game. Civ4 - great game, but 9.7/10? If you can only imagine 3% missing to absolute perfection then you have damn little imagination. And so on, and so forth.
Really, an honest rating should either allow > 100%, or say "90% if you do everything in the best way that I can imagine. Points above that only if you found better ways to do it, and because you had a few years to work on it that's not an unrealistic expectation".
Game Reviews: Obsolete? (Score:3, Interesting)
Video game magazines tend to be targeted at a different audience than me, and I'm not entirely sure what that audience is: maybe younger gamers, or those who aren't quite as involved. You get the opinions of one or two people, max, and possibly short scores from a handful of others that I don't know from Fatal1ty (Adam for the less hardk0re
Finally, before I plunk down money on a game I try the demo. Even a lot of console games now have computer versions with demos. They might lag behind, but I usually stay a year or two behind the console trend because the pricing on their games are ridiculous (some computer games are guilty of this as well, but they tend to be more reasonable).
Re:Aggregation not the problem (Score:2, Interesting)
I'd be interested in a website with reviews that only focus on the bad parts of the game. Advertisers, demos, and promotions be damned: list off the major problems, along with the little annoying things that can detract from enjoying the game fully, and I'll have a much better idea whether I should bother to play it. The good stuff is covered by pretty much every other site, so tell me about the lack of control, horrible camera, unintuitive menu, or repetitive gameplay so I don't waste my money/time playing it if any of those problems are dealbreakers for me.
Re:why? (Score:3, Interesting)
When i'm thinking about going to a somewhat iffy movie i'll check out RottenTomatoes and see what aggregate score it got. Then i'll look over the blurbs, keeping an eye out for any reviewers i tend to agree with. And if the movie has a generally negative rating i'll check out the negative reviews to see _why_ they didn't like it.
For example i was trying to decide whether to go see Underworld Evolution, at the time i checked RottenTomatoes there were only a couple reviews up and almost all rotten. However i noticed the following blurb: "Stylish, but doesn't really re-imagine the vampire genre, cluttering the screen with creatures that no longer shock or surprise us. It's also a bit too long to pack a mean punch."
Okay, i liked the first cause it had lots of werewolves and vampires beating up on each other and a cute girl wearing cool outfits, and i actually like longer movies. So the review was negative but for reasons that completly didn't apply to me, in fact the only part that really interested me was the one positive comment, that it was "stylish." So i went and saw it and was happy with the results. It was a fun werewolves and vampires beat up on each other movie with a cute girl in it.
Presumably if game reviewers write well thought out reviews that explain _why_ they liked or disliked a game then the same process will happen there once the situation settles down. People will look at the aggregate scores, but they'll also look at what their favorite viewers have to say and they'll look at why the negative reviewers didn't like it. Some people complained about Shadow of the Collossus being too short, but given that i don't have as much time as i like to play video games that's not really a huge concern to me. I go the game, enjoyed beating up the first four or so Colossi (while feeling sorry for them at the same time of course) and then got distracted with other games i got for christmas. Clearly that particular criticisim wasn't relevant to me but knowing what the criticism was was important.
Bizarre. (Score:2, Interesting)
They're pointless in the current climate. Personally, I decided to take this route because I was sick of seeing referral traffic from forums where folks had come along to the site, read the score (not the review text) and then promptly headed back to their favourite forum to trash us.
"Why did he give that 9/10? It sucks! Its not as good as !" etc., etc.
The alternative? Just give them the review. Then they don't have to ask WHY a reviewer who's spent a good while playing a game and a good while writing about it has given the score that he has, just because they're too damn lazy to read the review text. We have a short summing-up at the end of the review, so folks can read that if they really are adverse to checking out the rest of it.
All I know, is that if I've personally taken time out to visit a website to check out a review, I'm going to READ it. Not everyone is like that. Some folks do just want a number in place of valid reasoning and qualification of the opinion in question. Good luck to them. I guess they're the guys that return things to stores because they "didn't do what they expected."
Or the guys that stood in front of a 4ft tall black-on-white sign pasted on my local game store's window that said "We have 4 Xbox 360's and they are all reserved for preorders." on launch day, for three hours. Before walking up to the cashier when the doors opened and saying "Do you have any spare Xbox 360's? I didn't manage to preorder one. No? Are you sure? There are none left? Can you check?"
Re:Finally! (Score:3, Interesting)
This is where game reviewers could learn something from an established idustry that faced the same problem decades ago -- fine wines.
Robert Parker and other wine critics theoretically can assign values up to 100. But a 98 is a once-in-a-lifetime wine. A 90 is excellent (truly excellent). An 80 is fair.
Sure, most wines you'll see in the store are 85 pts & higher, but Joe BasementCask's '04 grenache isn't making it to market -- only the good, commercial stuff is.
But you know what? My wife wouldn't like a 92-pt cab, no matter how good everyone else thinks it is. Just like some people won't like CivIV, no matter how closely it approaches the ideal for that type of game.
My main point is, games should be reviewed on a bell curve -- like you say, scoring above 90 should be VERY difficult. And people need to understand that just because everyone who likes FPSs rated a game 9.2, doesn't mean they'll like the game that much, if they only play RTS games.