Forgot your password?
typodupeerror
Games

Review Scores the "Least Important Factor" When Buying Games 169

Posted by Soulskill
from the arbitrary-numbers-are-arbitrary dept.
A recent report from a games industry analyst suggests that among a number of factors leading to the purchase of a video game — such as price, graphics and word of mouth — the game's aggregated review score is the least important measure. Analyst Doug Creutz said, "We believe that while Metacritic scores may be correlated to game quality and word of mouth, and thus somewhat predictive of title performance, they are unlikely in and of themselves to drive or undermine the success of a game. We note this, in part, because of persistent rumors that some game developers have been jawboning game reviewers into giving their games higher critical review scores. We believe the publishers are better served by spending their time on the development process than by 'grade-grubbing' after the fact."
This discussion has been archived. No new comments can be posted.

Review Scores the "Least Important Factor" When Buying Games

Comments Filter:
  • Why (Score:2, Interesting)

    by ledow (319597) on Thursday November 26, 2009 @06:46AM (#30235504) Homepage

    And why? Because the grade-grubbing means that as of about 10-15 years ago, reviews are nothing more than adverts, and ratings are nothing more than auctions to the highest bidder.

    I've *never* bought any game because of a review. Not even back when they were a bit more honest (e.g. in the Spectrum days, it was very common to see sub-50% and even sub-10% scores of games, some of them were even immortalised in things like a "crap games collection"). Game preference is completely subjective and neither words nor pictures can convey how a game operates.

    But it's not just games that suffer from the problem - I know someone who buys cameras, cars, all manner of electrical goods etc. on the basis of the Which? review. I have seriously watched them buy something that costs a month's wages just because the Which? magazine said it was the best, only for them to discover that all the things *I* said about the brand / device / features etc. were true and it was useless to them. What was even more annoying is that they asked my advice every time about PC's and electrical goods, then completely ignored it, bought what the Which? review recommended, then complained and expected me to provide support for the thing they just bought.

    I read reviews as entertainment. If I want to know about a game, I might read the review of it to pass the time and introduce me to the *suggested* features that it may have. But I would never use them as a basis for a purchase... that's why you let other dummies buy it first and then hear first-hand from them after a month if they are still playing it and enjoying it.

  • I look at Gamefaqs (Score:4, Interesting)

    by tonycheese (921278) on Thursday November 26, 2009 @06:52AM (#30235540)
    When I personally buy a game, I look at gamefaqs user reviews instead of Metacritic. When looking at the main page of a game on gamefaqs, the first two averaged review numbers are exceptionally useful to me. They seem to give a very strong feel of what the general reaction to a game is - anything under a 7 is probably not worth my money. Also, user reviewers seem to me to play the games more thoroughly than someone who does reviews for a job, and game depth/replayability is a big point for me. Although, if I think about it, I generally buy games for Nintendo DS - price is pretty uniform and graphics can only get so good. In order to look up the game at all I had to have heard about it from my friends or some sites, so my experience doesn't really contradict his research at all.
  • by KermitTheFragger (776174) on Thursday November 26, 2009 @07:05AM (#30235626)
    If you ask people if they are willing to pay more for quality 90% will answer yes. However when the moment supreme is there to purchase for example a new notebook 80% will go for the cheapest and don't care about long term stuff like quality. I think there is a good chance this survey works the same; People SAY they are not influenced by reviews because 'Hey, I'm an original, I don't let anybody influence me'.
  • by clickclickdrone (964164) on Thursday November 26, 2009 @07:14AM (#30235660)
    >But we don't do we ?
    There is a reason for that. It's a lot of hard work and cost bringing a product to market and generally, the real dogs are killed long before they hit the shelves. I've been reviewing hardware/software for 20 odd years now and I can only remember giving a score of less than 4 a handful of times. Equally, 9 & 10 is rare (for me). The vast majority of stuff is 'good enough' and merits 7 or 8 out of 10. TBH, I get really frustrated by constantly dishing out 7s and 8s and the few times something has turned up for review that's truly bad, I'm been delighted as it gives me a chance to have a real opinion.
  • by SharpFang (651121) on Thursday November 26, 2009 @07:25AM (#30235710) Homepage Journal

    I have three tiers of deciding on purchase of the game.

    1. I read the review about what the game contains. I thoroughly ignore any "positive personal thoughts" about the game as marketing fluff. The negative ones do add to the value of the review but aren't all that important. I just read what is the concept of the game, and whether it is anything original, with potential - a good idea. If the review talks loads about graphics and sound and development time and prior franchise, even in total superlatives, it means the game is junk. A reviewer would concentrate on the really good points if it had any.

    2.I check some Internet fora to see what people complain about. If there is a number of complaints about the same thing, it may turn me away again. The thing being "awful execution of the wonderful idea" is one of possible choices.

    3. Then I grab the game off a torrent. After I'm through with it, I look back at how it felt. The only deciding factor is "I enjoyed it". Yeah, I enjoyed Stalker: Clear Sky, despite hopeless story, dull ending and reuse of content. I enjoyed Oblivion despite being dumbed down to knees level of Morrowind.

    If the game passes the three tiers of classification, I buy it.

  • by clickclickdrone (964164) on Thursday November 26, 2009 @07:33AM (#30235758)
    This is certainlhy an issue with US based sites/magazines. Over in the UK it's less driven by advertising spend (in my experience, at least). I've given fairly bad reviews to a few products and I still get new stuff from them to look at. Equally, I've had software from the US where they've asked outright if the review will be looked upon favourably if they advertisise with us. They seemed amazed that I was adamant advertising and editorial don't talk to each other. They can't, otherwise the whole point of reviewing is null and void.
    There is possibly an argument that because some firms let you keep the kit (sometimes quite expensive kit) and others always want it sent back, that this could affect your scoring but I try hard not to fall into that trap. That said, I often request review items I actually have a need for and this can actively work against it if it doesn't do what I'd hoped.
  • Amazon (Score:3, Interesting)

    by clickclickdrone (964164) on Thursday November 26, 2009 @07:37AM (#30235786)
    I used to write Amazon reviews - you know, the bit before the buyers reviews but after the manufacturers descriptions? I was impressed when I signed up that made it clear I could slate a product if it really wasn't any good. Their only stipulation was that I should suggest another product on their site that was know to be better. Seemed fair enough to me. I stopped some years ago but if that policy is still in effect, it owuld add some weight to their value IMO. This was the .co.uk - the .com had diffferent reviews and possibly different criteria.
  • by Your.Master (1088569) on Thursday November 26, 2009 @09:14AM (#30236248)

    You misread him. His argument was that after that effort, the ones that deserve 6 and below are cut and don't reach reviewers, and therefore don't exist.

    My personal opinion is that this means that he should rescale 6 down to 1, and leave 0 for what he now calls 0-5. If something is that incredibly rare, it doesn't deserve half the namespace to be allocated to differentiating between just how exceptionally bad it is.

    But he didn't say anything about sympathies to the devs.

  • by Alpha Soixante-Neuf (813971) on Thursday November 26, 2009 @11:36AM (#30237298)
    My answer to this is it's time to re-think your scale. Absolutely everything that goes sub 6 on your current scale should get an automatic 0. 0 means too far below industry standards to be considered a viable option. After that re-orient your 6-10 on a 1-10 scale. Nobody reading reviews gives a crap about the differences between a current 1 and a current 4. they are both equally unbuyable products and I see no reason for a reviewer to differentiate between them. However, there are lots of reasons to make small distinctions between a 6, a 6.5, and a 7 and I'd much rather have those difference given more weight so they exist on a 3-6 or 3-7 plane. It's all the little things reviewers know about that would help people make decisions. Are there other similar titles that do it better? Graphics subtleties, small control issues, bugs etc...

    Now it's frustrating because if you're the only one using a scale like this your reviews sound incredibly harsh, but to me there's no reason to give a spectrum at the bottom of the scale where the threshold for even considering buying it is way more like 4-5 at the lowest. To me a 1 should be the generic genre game that super fanboys will play and enjoy, but if you don't play like 20 games of the same style every year, then buy a different one. Then work your way up from there. There's no reason anything lower on the scale deserves it's own spot. It shouldn't be paid for under any circumstances and the fact that people can make games even worse than it doesn't mean the've accomplished anything either.
  • by fbjon (692006) on Thursday November 26, 2009 @12:09PM (#30237542) Homepage Journal
    I understand the rationale behind finding trusted reviewers, but I've never really bothered to go that in-depth. In fact, I do precisely what TFA says people don't do: look at the Metacritic aggregate. But not just the combined score, it's also important to consider the spread of the scores, and what kind of sites give the scores in any particular band: low, average, high.

    I find that when the spread is large, the extreme ends tend to point to piss-poor reviews that I can safely ignore. If there's a lot in either end, however, some of those are probably worth paying attention to. Most of the time, I look at a few "trusted" sites in the middle of the pack, such as IGN and GameSpot. GS invariably gives a lower score than IGN to any game, so I end up looking at that most of the time.

    Finally, if a game gets mostly high scores, the low score reviews tend to be informative, and vice-versa.

    So any time I buy a game or browse around a store, Metacritic is the first place I check, and then combine that information with the price of the game to arrive at a decision. So far the only times I've missed is when I didn't check Metacritic carefully.

  • by Forge (2456) <kevinforge AT gmail DOT com> on Thursday November 26, 2009 @01:11PM (#30238032) Homepage Journal
    This phenomenon is not unique to Games. I watched a TV Rerun of the Matrix last night and remembered that some Newspaper reviews were very harsh on it. That I would watch it again after all these years and loosing count of how many times I have seen it suggests that the reviewer has standards incompatible with my own. That is not the worse case however.

    There was the Mag Innovision letter to the editor after it's 17" monitor received the worst ranking in a roundup of 17" monitors. The Editor's choice award went to a Gateway 2000 monitor. The point of the complaint letter? "This is the same monitor, we just print different labels on the ones we ship to Gateway 2000." Or words to that effect.

    So as a general rule I have very little use for published reviews of any product. Word of mouth, and personal trials work best. Also it's good to know what advertisers are allowed to lie about.

    "This POS is the best on the market" -: Allowable lie.
    "This overpriced crap is great value for money" -: Acceptable lie.
    "This 500GB drive holds more data than 750GB of data without using compression" -: dangerous ground.

You are in the hall of the mountain king.

Working...