The Problem With Metacritic 131
Metacritic has risen to a position of prominence in the gaming community — but is it given more credit than it's due? This article delves into some of the problems with using Metacritic as a measure of quality or success. Quoting:
"The scores used to calculate the Metascore have issues before they are even averaged. Metacritic operates on a 0-100 scale. While it's simple to convert some scores into this scale (if it's necessary at all), others are not so easy. 1UP, for example, uses letter grades. The manner in which these scores should be converted into Metacritic scores is a matter of some debate; Metacritic says a B- is equal to a 67 because the grades A+ through F- have to be mapped to the full range of its scale, when in reality most people would view a B- as being more positive than a 67. This also doesn't account for the different interpretation of scores that outlets have -- some treat 7 as an average score, which I see as a problem in an of itself, while others see 5 as average. Trying to compensate for these variations is a nigh-impossible task and, lest we forget, Metacritic will assign scores to reviews that do not provide them. ... The act of simplifying reviews into a single Metascore also feeds into a misconception some hold about reviews. If you browse into the comments of a review anywhere on the web (particularly those of especially big games), you're likely to come across those criticizing the reviewer for his or her take on a game. People seem to mistaken reviews as something which should be 'objective.' 'Stop giving your opinion and tell us about the game' is a notion you'll see expressed from time to time, as if it is the job of a reviewer to go down a list of items that need to be addressed — objectively! — and nothing else."
But it's all subjective anyway. (Score:4, Interesting)
Re: (Score:1)
It really doesn't matter what scale you use or how you weight them. If you add enough similarly-weighted ratings together, the central limit theorem essentially guarantees that the result will have a normal distribution. So in the end, the mean is irrelevant; it's really only the (score-mean)/stdev that matters.
Re:But it's all subjective anyway. (Score:4, Funny)
Re: (Score:1)
For the love of Baal, please at least use <a href [xkcd.com], m'kay?
Re: (Score:2)
Does anyone else play the "Predict which XKCD this will be" game when Oblig XKCD links are posted?
(Those that don't know the strip numbers off by heart of course)
Re: (Score:3, Funny)
Re: (Score:2)
Does anyone else play the "Predict which XKCD this will be" game when Oblig XKCD links are posted?
(Those that don't know the strip numbers off by heart of course)
Every time. I don't have the numbers memorized, but I do recognize 'recent' vs 'not recent' and that helps sometimes.
Re: (Score:1)
Nobody rates lower than 0 or higher than 100 so it can't be a normal distribution because the tail would be cut off. This would primarily be a problem for large standard deviations or extreme scores.
Re: (Score:2)
The problem is a lack of agreement on what the median should be.
A normal distribution works when we know what the average value is (The peak of the distribution bubble) However we need a normal distribution on what the people think the normal is first to come up with the best median for the distribution of grades for a product.
Re: (Score:3)
Nobody rates lower than 0 or higher than 100 so it can't be a normal distribution because the tail would be cut off. This would primarily be a problem for large standard deviations or extreme scores.
Not just that, but there are numbers in the low range that just aren't going to get used. Who rates something as a 3 out of 100? What does that mean? It was absolutely terrible except for one relatively minor part that was done right?
Is that so? (Score:3)
This line of thought seems faulty, but I have to admit that I feel the same way for most scores reviewing sites and magazines deal out. A score of less than 7 out of 10 is reserved for seriously failed works, and typically these works never merit a recommendation. However, if anything below 7 is not worth experiencing, you essentially only have five possible scores: bad, 7, 8, 9 and 10. The rest of the scale is simply wasted.
I always wondered if this is caused by school grades in youth influencing what peop
Re: (Score:2)
Re:Is that so? (Score:5, Interesting)
I work for a review platform. We have decided that you only really need four ratings, Bad, Poor, Good, Excellent. We don't have a neutral option because really neutral tends to mean bad.
Of course, quite a lot of our users (and our marketing department) seem to prefer stars. Because an arbitrary scale is so much more useful that simply saying what you think of something. Apparently.
Re: (Score:2)
Might I ask what you review and for a link to the site (if there is one)? It would be nice to check out reviews using this philosophy.
Re: (Score:2)
I agree, I think your (the GPs) scale sounds sensible.
Re: (Score:2)
See also the San Francisco Chronicle's movie review "little man" [austinkleon.com]. Not a star or a thumb, but a very clear indication of what the reviewer thought. Is the little man applauding or asleep. Or is his little movie theater seat empty?
Stars are awful and should never be used. There's a reason YouTube dropped stars in favor of thumbs up/down. There was a serious bathtub curve on the stars, putting a huge percentage at 1 or 5 stars, and only a few 2,3 or 4 stars.
That's vastly different than how I use stars. But w
Re: (Score:1)
Does every one of your reviews give the 4 different possible values - in correct order - and then list what the reviewer gave the current game? Without that, having a word like 'Good' is so bland/overused as to be meaningless. Context matters, and stars give a much better, immediate view of that kind of rating.
Of course I hate that most places disallow zero-stars :p
Re: (Score:2)
I send a friend with a similar taste in films a guide to films I've seen:
- must buy
- must see
- worth seeing
- ok
- crap
- holy shit it was bad
Some of the 'must see' films aren't necessarily great films, but how can you love cinema and not watch something like Breaking the Waves.
Metacritic on the other hand is an excellent way of finding professional and individual reviews, and I use those as much as the score to choose which games to buy.
Re: (Score:2, Offtopic)
We often think of using the school level of grading.
60 F (0.0)
60-69 D (1.0)
70-79 C (2.0)
80-89 B (3.0)
90-100 A (4.0).
Of course school grading has gotten corrupted to just mean. Less then C you don't go to college. C-B means you can go to a normal college, A you go to a well known college.
However it should be the following...
F is a Failure where they should take the class over again.
D They can advance to the next class... However they are at high risk of failing the next class, you should have the op
Re: (Score:2)
Sure they can be mapped to real numbers but lets face it 5 and below are not useful to me. If a game gets an F, then I am probably not going to play it. A score of 1 is almost unreasonable...its like they managed to ship me a bla
Solve it with Machine Learning (Score:2)
"Trying to compensate for these variations is a nigh-impossible task" - definitely not.
Re:Solve it with Machine Learning (Score:5, Interesting)
Your description is not so much machine learning as basic math. If you just want each scoring system to have equal weight on the results then compensating for the variation is trivial.
Where machine learning would come in is to find underlying pattens in the data.
This could be used to weed out reviewers to lazily copy scores, or are subject to influence.
It would also allow them to test your scores for some games against the population of reviewers to find reviewers with similar tastes.
You could also use clustering algorithms to find niche games which got a few really strong scores but whose average was really pulled down because they don't have wide appeal.
Just use a bell curve (Score:5, Insightful)
Re:Just use a bell curve (Score:4, Insightful)
Re: (Score:2)
Bell curve doesn't work that well either... (Score:5, Insightful)
One reviewer might only rate highly hyped games which he expects to be good (nearly all fall to 60-100 range) and other reviewer tries out pretty much everything he encounters to find out those lone gems among less well-known indie games, etc. (let's say ranging from 20 to 95). We can't just take a bell curve of each and say "Game A is slightly above average on first reviewer's scale and Game B is slightly above average on the second reviewer's scale... so they're probably about equally good!". Sure, with large number of reviewers, you can still see which games do well and which won't but you have lost at least as much precision as you would have if you hadn't taken the bell curve in the first place.
That said, I don't know if reviews are that relevant anymore. I am active gamer but don't remember when was the last time I read a full review... There have been two times recently when I bought newer games from series I had played years ago (Cossacks and Anno 1602). I just wanted to take a quick peek on whether the games were considered about equally good, better or worse than the ones I had liked and whether they were very similar with just better graphics etc. or if some major concept had changed. That consisted mostly of looking the games up on Wikipedia and quickly glancing the first reviews I found using Google. I think I also checked the metascore, but it was more among the lines of "I'll buy it unless it turns out to have metascore under 60 or something". I didn't use that as exact metric.
Most games I buy are ones recommended to me by my friends, those recommended by blogs I follow (e.g., the Penny Arcade guys' news feed... you could consider those reviews, but they don't mention the games they hated, don't give scores, etc., just mention "Hey, that was pretty good. Try it out.") or those that just seem fun and don't cost much (When I noticed Orcs Must Die on Steam for under 5 euros, I didn't start doing extensive research on the critical acclaim of the game.)
Re: (Score:2)
That sounds like a pretty good idea, actually (Score:2)
That sounds like a good idea. Give me a moment and I'll patent it.
Seriously though, that idea is mostly good but I think that the passing of time might be a problem. Deus Ex was awesome game when it came out, but if it were to come out today it wouldn't be all that good because the video games have taken huge leap forward in the last 12 years. So, should I vote it worse than nearly all new games and completely ignore the context, how much it influenced the genre, etc.? In general, it's very hard to compar
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
When a tremendous amount of the scores are all 98, a 5% variation would be a good thing.
Re: (Score:3)
Re: (Score:2)
And then the average reader of Metacritic doesn't understand how the score is calculated and cries foul every single time.
Yeah, that'll work.
Re: (Score:2)
Or perhaps you are the 1 in a hundred person who hates Oblivion when everyone else, myself included thinks the game is brilliant, I think it was more fun than Skyrim (fights too balanced and orchestrated - you can always win).
No, of course you're right, the rest of the world is delusional and millions of people just imagined having great fun playing Oblivion.
Why do you hate it anyway?
Re: (Score:2)
If boring, repetitive quests and lack of challenge is your thing, sure you'll have a great fun playing Oblivion.
So you didn't do the Mages Guild quests then, or the divines quests, a lot of uniqueness there. If you rushed to complete the game's main storyline then you're doing it wrong.
So if Oblivion is not interesting then what single player game is?
Re: (Score:1)
I too found Oblivion to be boring. Once I set foot outside the starting dungeon I felt lost, without sense of purpose or direction. I liked skyrim a whole lot more due to its faster pacing and sense of urgency. The improved graphic fidelity also helped a lot. Other open world games that worked for me were the new Fallout series. Another example of an interesting single player game is obviously (the golden standard) Baldur's Gate 2, where none of the encounters are scaled and everything is unique.
Re: (Score:2)
Oblivion was boring. Morrowind was not. I did Mage's guild, Fighter's guild, Dark brotherhood (the high point of the game), and the main story line my first play through (and more, I'm sure, but it's been several years and that's what I remember). Did not even slightly compare to Morrowind in terms of expansiveness, creativity, or fun (with the exceptions that Dark Brotherhood was great and they fixed alchemy, although in Morrowind you could pretend alchemy wasnt' horribly broken if you wanted).
Morrowind
Re: (Score:2)
While I never played Oblivion, I recall the buggy mess that was its predecessor Daggerfall, so I understand the feeling. However, metacritic did help illustrate in stark contrast what a racket those gaming publications ratings were vs what the gen. public thought(I'm looking at you steamy pile of s
Hmmm? (Score:2)
Metacritic a scoring system that works for a lot of people and seems to work for them isn't perfect? News at 11. Really? So someone writes an opinion piece on it backed with opinion and this is interesting? The methods Metacritic use seem to be fair and work, so who cares that someone doesn't think they are perfect?
I'll use the numbers as a guideline, but not as fact. Just like wikipedia, it's a place to start. Now if the article was aimed at pointing out that the publishers put too much emphas
Re: (Score:1)
Re: (Score:2)
It just seems to me that the article under question wasn't trying to point out that people should use the Metacritic score with some reservation, instead it seems like it was just an attack on metacritic itself complaining that it has too much influence. If the article would have approached it in a different matter, e.g. because of people's blind belief in metacritic producers of software are more interested in a high metacritic score than about the product itself, but here are reasons the Metacritic scor
I don't get the point of it anyway (Score:4, Insightful)
It can be used as an executive summary (Score:5, Insightful)
I find it useful for that. If there's something you have little knowledge or information about it can give you a quick breakdown of what you might expect. For example if a game has a 90 metascore, you know it is something that you should probably look in to further, that is uncommonly high. If something has a 40 metascore, you can pretty much give it a miss, that is uncommonly low.
What it'll then get you for things you do want to look in to further is a list of reviews. So you can see what sites have reviewed it, and then go and read the specifics if you wish. Along those lines it is a quick way to find good and bad reviews. When I'm on the fence about something I like to see what people thought was good and bad. I can then weigh for myself how much those matter to me.
Average ratings really can be of some use to filter. I just don't give enough of a shit about every game to go and read multiple full reviews on it and research it. So if it isn't a game I was already interested in, I want a sort of executive summary to decide if I should give it any more time. Metacritic helps with that.
Two recent examples:
1) Endless Space. I had never heard of this game, an indy 4X space game apparently, though rather well developed. Ok well ambitious indy titles can be all over the map. Metascore is 78. That tells me it is worth looking at, it is on my list and I'll look at it more in depth when I feel like playing such a game.
2) Fray, a turn-based strategy sci-fi game. Again, something I hadn't heard of, however a kind of game I like so maybe I'd be interested. Metascore of 32. So no, not wasting time on that.
Other games I won't bother on the Metascore, just use it to find reviews. Like Orcs Must Die 2. Looking forward to that one, so I'll spend time researching it to see if I want to buy it. I liked the original enough it'll be worth looking at reviews, no matter what the score, so see if I think I'll like the next one.
Re: (Score:2)
Orcs Must Die 2 is a must buy - it's on sale 10% off right now too ;) (I LOVED the first one)
Related to the article:
It's just retarded. XKCD aside, it's worrying about the subjectivity of converted numbers when the initial number is completely subjective as well. If a reviewer is giving something say 3 out of 5 stars, maybe he thinks it's 3.12658 stars out of 5, maybe 2.9674... who cares!
Re: (Score:1)
As a buyer of Endless Space, be careful. It's like a formalistic, spreadsheety MOO2 without combat, without charm, without espionage and poor diplomacy, and no real racial picks.
This largely comes from ship design. There are three types of weapons, and they work like this: Beams 1 (100 damage), Beams 2 (200 damage), Beams 3 (400 damage), Beams 4 (600 damage), Beams 5, (1000 damage).. all the way to something like Beams 8. No special systems. No range triplers, no MIRV missiles, no black hole generators.
Warl
Not broken, does exactly what it intends. (Score:2)
A 80 game is still, when ranked by a buch of people, better than a 70 game. On average.
You can do all kinds of statistical analysis on each reviewer for each platform and each genre of game and make adjustments to them, but at the end of the day you still put the pig in the sausage grinder to get a single number. And that number will not be significantly different than what's there now and the ranking of games will not change.
Compare Metacritic's movie review numbers to Rotten Tomatoes'. They both use dif
Re:I don't get the point of it anyway (Score:4, Funny)
Just because 10,000 out of 15,000 like something, it has no bearing over if I will like it and quite often leads to the oposite.
So you're not interested in my remake of Twilight staring Justin Bieber?
Re: (Score:1)
Well DUR? (Score:2)
Of course it can be wrong, but its a good indicator of what others think of the game.
Nothing on earth can tell you if YOU will like the game unless you play it.
I personally sometimes enjoy playing terrible games, (or games with terrible reviews) and find them quite charming.
Re: (Score:2)
And in fact, there are systems just like this for finding music which suits your tastes.
Who Cares? (Score:1)
They shouldn't worry, Slashdot beats Metacritic hands down for subjectively erroneous (mod) scoring.
Mod me up for the hell of it.
tnx.
stupid article (Score:2)
Depends on what you mean by using the range (Score:4, Interesting)
In most US schools, the scale is:
A: 100-90
B: 89-80
C: 79-70
D: 69-60
F (or sometimes E): 59-0
So while you can percentage wise score anywhere from 0-100 on an assignment and on the final grade, 59% or below is failing. In terms of the grades an A means (or is supposed to mean) an excellent grasp of the material, a B a good grasp, a C an acceptable grasp, a D a below average grasp but still enough, and an F an unsatisfactory grasp.
So translate that to reviews and you get the same system. Also it can be useful to have a range of bad. Anything under 60% is bad in grade terms but looking at the percentages can tell you how bad. A 55% means you failed, but were close to passing. A 10% means you probably didn't even try.
So games could be looked at the same way. The ratings do seem to get used that way too. When you see sites hand out ratings in the 60s (or 6/10) they usually are giving it a marginal rating, like "We wouldn't really recommend this, but it isn't horrible so maybe if you really like this kind of game." A rating in the 50s is pretty much a no recommendation but for a game that is just bad not truly horrible. When a real piece of shit comes along, it will get things in the 30s or 20s (maybe lower).
A "grade style" rating system does make some sense, also in particular since we are not rating in terms of averages. I don't think anyone gives a shit if the game is "average" or not, they care if it is good. The "average" game could be good or bad, that really isn't relevant. What is relevant is do you want to play a specific game.
Re: (Score:2)
South Africa uses a different set of numbers, so the same grade letter means something different for us, and probably something else completely for other countries too.
Re: (Score:1)
Re: (Score:2)
I find it very important to think of games this way because technical incompetence (much like failing comprehension or ability with grades) is such a large part of gaming. There needs to be that large bottom half of the scale to allow for certain technical failures, which render the g
Re: (Score:2)
It's called the Four Point Scale [tvtropes.org] for precisely this reason. Professional reviewers are very limited on how severely they can criticize the faults of a game when it comes from a big publisher.
When the critic reviews of a AAA game are all 80+, but the user reviews are in sub 4.0 red text, tread carefully.
A score without a review is useless (Score:1)
I'd never, ever let a metacritic score determine whether or not I buy a game. That's not to say I don't let reviews (and review scores) influence a purchase, but I find metacritic useless.
When I check reviews of a game to work out whether I want to buy it, I'll look at the score, but it's only one small factor. What I'm actually looking for are certain factors that might be picked up in a review that will be highly likely to influence whether or not I like a game.
For example, I hate - and I do mean absolute
We need a meta-meta-critic (Score:2)
Re: (Score:2)
Our game wasn't perfect, neither was it crap. It is fun, addictive, beautiful
That's completely subjective.
But was it a 2 or a 10?
Apparently to them, it was.
Re: (Score:2)
Well then, you could argue that their opinion isn't very relevant, if they only scraped the surface of the game. Reviews are for those who want to shell out their hard-earned cash on a game, and they probably want to go a little deeper.
Re: (Score:1)
But was it a 2 or a 10?
Apparently to them, it was.
Sure it was, but should they be at Metacritic? Anyone can rate anything without looking if they like to, but it will be of little or no use to anybody else.
Give later reviews more weight.... (Score:2)
a)The reviewer didnt actually spend enough time with the game to give it a meaningful review and/or
b) the reviewer had access to the game early, which of course raises questions about objectivity.....
The best reviews IMO are those that come out at least a week after the games release....
Re: (Score:2)
Well, it's not as though reviewers get the game the same day we do and then put them out the same day. They get them often weeks in advance, and save for really open ended games like Skyrim, most games can be thoroughly played in the window the reviewers are already given.
Re: (Score:2)
I should also point out that just because reviewers are given games early doesn't raise many questions about objectivity. It's pretty much how the entire industry functions. The reviews you need to take with a grain of salt are the pre-release reviews that come out some time before the game actually launches. Publishers often impose restrictions on what you can say before release, but they have no control over what is said after release. As long as the reviews come out the day the game launches, there shoul
Re: (Score:2)
5.5 rounds to a 6 - PASS! (Score:2)
And that is how maths for students works :-)
The average is never to be taken at face value (Score:2)
Diablo 3 on Metacritic is the 2nd highest rated current game.
Don't take averages for truth, they're just averages. Use Metacritic as a source of reviews, find the reviewers (people) who you have the most affinity with over time, and then focus on what their own scores are.
Re: (Score:2)
Sure, but if you look at the user scores, it's also in the lowest 5-7.
http://www.metacritic.com/browse/games/score/metascore/year/pc?sort=desc&year_selected=2012 [metacritic.com]
Now, if there were only 10-15 reviews I can understand that this number may not be accurate... but generally the fans don't like the game and that tells me it's probably not worth playing, especially Blizzard fans who are usually not critical to the mothership. Currently, the majority of user reviews are all negative, by almost 2:1.
Re: (Score:2)
User scores are not reviewer reviews.
Reviewers have experience and thoughtful analyses but suffer from small sample size and conflicts of interest.
Users have the strength of numbers but suffer from groupthink and emotional coloring.
The way I use MC is that I get a good feel for the game based on user score averages, then look at the reviewers that I like and analyze their pros and cons. Then I make a decision, based on whether the cons are bad enough for me or not.
Re: (Score:2)
User scores are not reviewer reviews.
Not sure where I said they were...
Reviewers have experience and thoughtful analyses but suffer from small sample size and conflicts of interest.
Users have the strength of numbers but suffer from groupthink and emotional coloring.
In an entertainment medium I find emotional coloring an important metric, especially for companies that have such a huge following as Blizzard with such a rabid fan base. There are a few fans, but a majority are not happy with the game.
Professional reviewers have ad revenues to look into. If they continually rate EA games poorly, EA will pull it's advertising dollars from their income. Of course they are bias and there have been multiple... multiple stories and articles
It's good for avoiding rotten apples (Score:2)
Metacritic is good for avoiding games that are complete crap. Other than that, you really have to read some of the reviews to decide which game you will like more.
I just played The Last Story (metascore 82) after buying it instead of Xenoblade Chronicles (92). After reading some reviews I was sure that I would prefer The Last Story even though Xenoblade has a greater score and the games are in the same genre. Xenoblade is usually praised e.g. for having lots of stuff to do, but I really wanted a game that I
Use it for Good, and not Lazy (Score:2)
There are links to all of the collated reviews. They're there so that you, the consumer, can perform your due diligence more easily. Personally, I like to scan the middling and bad reviews to get an idea of the kind of warts I should expect if I buy it. One reviewer's selling point ma
1UP moans that everyone is is doing it wrong. (Score:2)
Rottentomatoes (Score:5, Interesting)
By that logic, Rottentomatoes (which averages reviews using only a binary fresh/rotten scale) should be utterly useless. Except it isn't. It's IMHO the most dependable rating site on the net.
It seems the magic lies not in the rating resolution, but in the quality and size of the reviewer pool (100+ for Rottentomatoes). In other words, make the law of averages work for you.
Re:Rottentomatoes (Score:4, Interesting)
Re: (Score:2)
How is that a good metric? It's _terrible_ granularity. I count count the number of movies I was so-so about and had no strong opinion one way or another. And Rottentomatoes forces a "HELL YES!" or "HELL NO!" response with the thumbs up/thumbs down. Hell, if I'm just not in the mood one day for an action flic
Re: (Score:3)
Rottentomatoes rating is not a rating of how good a movie is. Rather, it is how likely you are to enjoy it. It is a probability!!
A movie with 10% on rottentomatoes doesn't mean its a movie worth a 10 grade, it means that only a niche audience enjoyed. So you're less likely to be part of that 10%, but its absolutely possible you still love the film, for very specific reasons. Similarly, a movie with a 98% rating isn't necessarily the best movie or a very high quality film
Re: (Score:2)
Look at how game reviews work though. Unless the game is a complete debacle, every game will get a positive review, and the game would have a "fresh" rating of 100%
Scores are unnecessary (Score:2)
Yeah, that's why Rockpapershotgun have failed...oh wait.
That doesn't count their gamma rating, of course. (Score:1)
Hmmmm...I hadn't realized this issue with Metacritic before. I now rate Metacritic's quality as 4.7 kumquats, down from 2-8 exahogsheads per quadriliter :(
As a game reviewer (Score:1)
As a ten+ year game reviewer (shameless plug: game-over.net), I see the problem from the other side. Even on a single review board, there are variations in how "hard" individual reviewers score. Over the years we have tried to implement a scoring system, giving XX/20 for graphics, XX/20 for story, that kind of thing, but found disagreements among reviewers as to the weightings. Is good graphics equally as important as good plot? What about good music and sound effects? What one reviewer sees as retro,
truly (Score:2)
The problem with metacritic is that they don't take into account the Law of Truly Large Numbers.
(I don't know what that means, but I've decided I'm going to say this in all discussions involving statistics).
Works when people are sane (Score:2)
If your going to use some crazy scheme to rate games, then perhaps those websites should provide some kind of a translation value so that Metacritic can correctly identify the intention of the crazy review schemes.
But ultimately if you are getting reviews from 100's of websites, the aggregate value should be fairly accurate. I think a game that is rated 50% is bad compared to a game rated 90%, but I don't think people really care about the perceived quality between two games if their ratings are like 85% a
Another problem (Score:2)
Larger/louder/more voices drown out smaller/quieter/fewer voices -- regardless of the authority or quality of comment. (Unlike on /., which has moderation and meta-moderation based on content.)
True story: My sister and brother-in-law left their kids with their grandmother and escaped to see a movie and relative peace for a few hours. My sister came back, really angry with her husband. "But sweetie," he said, "_2012_ got a good score on Metacritic!"
Really. Happened.
Value in variation of reviews, not the "score" (Score:3)
I think the value in metacritic isn't the "score" but the variation across all reviews. You could have two titles with identical "80" scores, which would otherwise indicate both titles are equally well liked.
That being said, one title could have all of its reviews be between 70 and 90, while the other could have a lot of low scores and a lot of high scores. The high variation in scores tells you that there's something about that title that's amiss.
It would be interesting to see statistics compiled for reviewers, too. Do some reviewers always deviate above the average? Below? I would think a reviewer with a higher variability of ratings would be more trustworthy than one who was consistent with their reviews.
The problem is a completely different one (Score:2)
It's not one about 100 to 0 or A to F. It's one where people are hardly objective or, rather, how people rate games.
Take a look around. Games will get a rating of 90 to 100 if they're really good, 80 to 90 if they're halfway decent and 70 to 80 if they're kinda lukewarm. Then there's a big nothing until the 0-10 bracket for the stinkers. WTF?
This doesn't make any sense at all. But that's how things run today. Every game maker presses to get a 90+ review. Even if the game doesn't really deserve it. What is a
Surprise! (Score:2)
Oversimplifying things leaves them... what's that word? Right. "Oversimplified".
Who could have guessed that?
What's the problem? (Score:2)
I don't have a problem with it. (Score:2)
I've found Metacritic's scores to be pretty good when it's pulling from a large number of reviews. It's further enhanced by the inclusion of a separate user score. Score inflation is a problem with nearly all reviews so it isn't like Metacritic is really suffering from any inconsistency. I think at this point your average person is well aware of that and assesses scores accordingly.
My problem isn't so much that most scores float above 75 except when they're exceptionally bad. My problem is with blatant infl
Re: (Score:2)
I also tend to pick out reviews across the scoring range to get a more detailed assessment.
I do that too. Focussing on the raw headline average score is a crude way of using Metacritic and people shouldn't be surprised that it leads to crude results.
It's also interesting when you're looking at 1-2 year old games how many games have a lowish score because of release day issues. Buying the game fully patched for half the price means you can get a game that were it released in today's form back then would've got 10-20 points higher on the average, and is in fact an excellent game.
There are some game
Somewhere, Adam Sessler is swearing. (Score:2)
use slashdot's poll caveat (Score:1)
"This whole thing is wildly inaccurate. Rounding errors, ballot stuffers, dynamic IPs, firewalls. If you're using these numbers to do anything important, you're insane"
Transcript please (Score:2)