Game Reviews are Broken? 168
Kotaku is running an opinion piece looking at the process of reviewing videogames, and comes to the conclusion that the whole system is entirely broken. Author Mark Wilson takes potshots at the concept of assigning a numerical valuation to a game, and the emphasis on product reviews rather than content reviews. "If there is no such thing as a perfect game, when why the hell are you scoring out of 100? It's not just PC Gamer that thinks this way--most publications, even those who do give out 'perfect' scores, do so begrudgingly. It's as if the developer has somehow cheated and broken their system. The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie didn't imply 'perfection' but supreme excellence. In most cases, games are penalized through being divided by a sum that they can never possibly reach."
Not New (Score:5, Insightful)
Game publishers, consumers, and even the reviewers themselves have been going on about the shortcomings of the current system for quite a while now. Yet we never see any alternatives being proposed. I say to the article writer, "Yes, I agree that the current system sucks. But what is your alternative?"
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
You see, there's no such thing as a perfect restaurant, either, but there -are- 5-star restaurants. That's 5/5 on the game scoring chart. Or 10/10. Or 100/100. It's all the same thing.
Just like a restaurant, 10/10 for a gaming score simply means 'it's as good as could possibly be expected' or 'it's better than expected, even when you expect excellence.' Nobody claims it means perfection anymore since t
Re: (Score:2)
Rating on a percentage (or equivalently, a 10-point scale with one decimal place) is really only good for wanking around at the top of the scale, making sure that X game has a higher rating than Y, which is better than Z (totally independent of the kegs, gadgets, or bags of money, right?). That's what gets tossed out with, for example, a five-star system. Is it really important to me whether
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Halo 3 is a good example. Multiplayer isn't important for me. If I had ju
Re: (Score:2)
What you seemed to miss with this story is that it is impossible to rate something like a game with such a high precision.
Wrong. What both you and the article writer are missing is that game ratings are not and never were intended to be any kind of absolute data point. They're relative. It's the whole point of rating games. If a reviewer gives a game a rating of 88, it means he thought the game was very good. Not quite good enough to qualify for an "Editor's Choice" ("9-star")rating, but still very good - and noticeably better than games rated at 80 (or an "8-star"). He also thought it was not quite as good as past games his
Re: (Score:2)
And some review sites already DO have multiple ratings for a game. GameSpot has their own, their readers', and other critics all listed at once. Reader reviews almost always score the game higher than GameSpot does, meaning that GS looks for a little more quality at the top end of the scale than their readers do.
Re: (Score:2)
Re:Not New (Score:5, Interesting)
Am I the only one who finds that comment just odd? While I can agree with you that the system is broken because there's no such thing at a "zero" rated game, but I do not see the difference between 100 points and 4 stars, besides it being simply "divided by 25". Then, of course, that's not good enough, so they start assigning 0.5 stars.
Also, I never considered "100/100" to be perfection, but as "supreme excellence" as noted. After, WTF is the difference between "supreme excellence" and "perfection"? Someone is just trying to argue semantics.
Of course, I don't even like the "four star" or 100 point numerical system. When I ask/tell people about a movie I simply say "Is it worth seeing in the theater?", "Is it worth a theater matinée?", "Is it a rental?" or "not worth your time, period".
In this sense, I saw the "Number 23" in the theater and I recommended that it was worth seeing in the theater. While "28 Weeks Later" was easily worth waiting for a rental (despite being a fan of "28 Days Later" and zombie films in general).
In that regards, I would say games should be rated as "buy it!", "rent it/demo it!", "stay away". (rent for console / Demo for PC games). Guitar Hero games are "buy it" games while something like Zelda:Twilight Princesses might be a "rent it" kind of game (I bought it, I'm a fan of Zelda, but still feel money better spent on a rental. I would not have given it 100/100 as some reviews did). World of Warcraft? "Buy it".
Of course, such a system needs a context. I'm not going to tell a FPS fan to buy an RPG, it's in the context of RPG fans.
Cheers,
Fozzy
Re: (Score:2)
If you spend time just enjoying the ambience of that game then it would presumably cost you far more to rent it? I spent a few hours one night just fishing.. definitely a 'buy it'..
Re: (Score:2)
Re: (Score:2)
While there's no difference between 100 pts and 4 stars, there's a world of difference between a 75 and 3 stars. Generally speaking, video games don't score under 70 or so unless they're complete crap.
Then again, that doesn't really make the rating system broken since we can make sense of it. The
Re: (Score:2)
According to the PC Gamer scale, 60-69 is "Above Average." 50-59 is "Merely Ok." So not complete crap by any stretch.
Re: (Score:2)
Re: (Score:2)
And this part of your comment shows what's already broken with the game rating system. Th
Re: (Score:2)
I guess my complaint is that a decent conversion from percentages to stars would need to subtract 50 from the scores, and assign half a star for every 5 points thereafter. The scores are grossly skewed and overinf
Re: (Score:2)
3 stars - These are movies that were good to watch and I'd recommend watching them if they interest you at all or if you have similar tastes as me.
4 stars - Basically a shortened list of my 3 stars but these movies are just plain better. They are rewatchable, have interesting and new ideas, or might just be interesting visually.
5 stars - The best movies I've seen and if you haven't seen the
Re: (Score:2)
Re: (Score:2)
Maybe (Score:5, Informative)
Although he has a point, we humans love to compare and if you don't give us any metric by which to do that, then we don't feel like anything has been achieved.
"SuperGame is really good" is meaningless to me. What I want to know is, is it any better than GreatGame? If the reviewer gives a score for both then I can understand which he/she feels is better and by what margin. Since I've played GreatGame (and assuming I trust the reviewer), then I can set some sort of expectation of what SuperGame will be like.
Personally, I use Metacritic [metacritic.com] which aggregates a number of reviews. Again, it's not perfect, but when it gets a 75 or above score, I can be reasonably certain that I'm not getting a dud game. It might not be my type of game, but if it is, then it shouldn't be disappointing.
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
Except that the scores aren't derived from any kind of objective framework, so comparing the scores of two games really isn't meaningful. Maybe the reviewer's wife isn't giving him any that week and he's ticked off, so all his scores are 10% lower. Maybe his favorite American Idol contestant won that week and all his scores are 7% higher. The scores aren't scientific; they're completely subjective
Re: (Score:2)
Re: (Score:2)
I didn't even have to go to TFA for that, grabbed it right from the /.
Re: (Score:3, Interesting)
Yup, I find metacritic quite useful (I think gamerankings is another site among those lines), however I think that a service similar to what Criticker [criticker.com] provides for films would be good. In criticker you put a score to serveral movies you h
The best review (Score:5, Insightful)
The best review has no score. Simply somebody playing the dang game, and talking about what they like, what they don't like, what they'd improve, what really bothered them, what really excited them.
Find a reviewer with a decent command of the language, and who likes the sorts of games you like, and you're good to go.
Re:The best review (Score:5, Informative)
This is why I like the Zero Punctuation [escapistmagazine.com] reviews so much. Yahtzee has a decent command of the language, goes through all of the good and bad parts of the games, and gives a quick conclusion stating his opinion of the thing.
O RLY? (Score:2)
I thought we all liked Yahtzee because he's fucking hilarious.
Re: (Score:2)
Re:The best review (Score:5, Insightful)
As TFA states, the review industry is necessary because of the large amount of games coming out and the large proportion of crap that inhabits it. If something is crap from end to end and at least four different people agree, then there's no need for me to look further.
Re: (Score:2)
The problem I have with that is that around number 2000 or so, the reviewer is knocking points off for 'there's nothing here that we havne't seen before.'
I don't care that the game is very similar to the other several RTSes, say, that came out that month; I'm interested in that game as that game.
Re: (Score:2)
Re: (Score:2)
Originality as compared to what?
Lets say that an RTS comes out, oh, generic science ficition. Then, a few weeks later, using the same engine, a different studio puts out an official Licensed version of, say, Lexx: The RTS.
The reviewer likes the first game, but nails the second game for being 'game one, only with differnet names and unit models.'
Somebody who really like Lexx isn't going to care about that; they want to play the Lexx game, and the fact that it's 'nothing new' compared to Generic Scienc
Review Score As a "Fact Crutch" (Score:2)
A good example (Score:2)
Re: (Score:2)
Re: (Score:2)
Really, scores and the like are such a small part of it all that suggesting everything's broken because of them is a bit of a whinge IMO.
Re: (Score:2)
As a game developer, I agree that a sim
Exactly... (Score:5, Insightful)
Re: (Score:2)
This is sort of my take. Halo 3 is a definite example. I know it's supposed to be good, but it has some real flaws that should prevent it from getting a perfect score. Many magazines and such seem to operate on the Famitsu model (as I had it explained to me). Some games just get high scores because they are fun an are expected to (say FF: XIII will). The more obscure games get real ratings.
At this point, while I read other reviews (and use sites likes Game Rankings), I really like X-Play. They tend to do a
Re: (Score:2)
The unfortunate thing is that many sites/mags aren't trustworthy. It's one thing to give Halo 3 a 5/5 when you point out some flaws and weaknesses but say that the game is fun and a real blast in multiplayer. But I've seen many reviews of games (other games, I haven't read much on Halo 3) where reviews just pass over that kind of stuff because it seems that not what people want to hear.
I agree with Halo 3. It had very poor graphic sin the cut scenes. It was visually blah, fun single player but short, and multiplayer is fun but not all that new. If I was a game review site it would be a solid 3/5 or 8/10. Fun, short, good multi-player, and remarkable mostly for it's hype and sales.
Re: (Score:3, Insightful)
Re: (Score:2)
That's the double edged sword. I play a decent number of games, and follow quite a bit. If I read the control scheme is like that of Metroid Prime 3, the story well told like Psychonauts, and the often feels like SSX... that means something to me. But if you haven't played those games (or some that are very similar) than that review is nearly meaningless to you.
Writing that little sentence made me think of something else though: many games get compared to their previous selves. All the magazines compare th
Re: (Score:2)
These very thoughts came to my mind when I was reading all the Halo3 Reviews... When a game has so-so single player and awesome multiplayer...how does that get the game loads of perfect scores? A perfect game wouldn't need to make up for areas of lacking ANYWHERE. That aside, even the multiplay, while fun, is far from perfect. Halo3 was, and is, a great package but it's nowhere near a "perfect" game. I'm not just picking on Halo3 here either. HL2 for instance was a phenomenal game....but to call it "perfect" (like so many reviewers did) is just naive and downright inaccurate.
The last perfect score I remember seeing was for Soul Caliber on the Dreamcast. Gotta say, it is perfect. The only possible quibble I can come up with is that they could have had maybe a dozen voice samples for the announcer to cycle through to avoid repetition. That's about all I can fault it on! Fucking 10/10, absolutely.
I'm playing through Oblivion right now. That game is easily, easily a 9/10. It really lives up to the hype and deserves to be one of the top-rated games on console and PC. However, there
Re: (Score:2)
I'm playing through Oblivion right now. That game is easily, easily a 9/10. It really lives up to the hype and deserves to be one of the top-rated games on console and PC.
And that demonstrates the uselessness of numeric-only scores. I'd give it a 4/10, max.
That doesn't tell anyone anything other than you liked the game and I didn't. My 4/10 doesn't relate that I thought the game was a $50 GeForce stress-test, only slightly less entertaining than clipping my toenails; that the leveling system was stupidly broken, the storyline a snoozer, and the "openness" of the world including beast level-scaling was more a bug than a feature since it contributed to the stupid brokenness
Re: (Score:2)
And that demonstrates the uselessness of numeric-only scores. I'd give it a 4/10, max.
That doesn't tell anyone anything other than you liked the game and I didn't. My 4/10 doesn't relate that I thought the game was a $50 GeForce stress-test, only slightly less entertaining than clipping my toenails; that the leveling system was stupidly broken, the storyline a snoozer, and the "openness" of the world including beast level-scaling was more a bug than a feature since it contributed to the stupid brokenness mentioned above, and made it a lot boring a lot sooner since there was no drive to visit new places now that you were strong enough.
And that's why the rest of the review is there to explain where the numbers came from. If there's one thing I hate more than anything that exists even in good games, it's checkpoint saves. I hate the thought of having to play through the same five minute sequence over and over until I satisfy the damn checkpoint. Supporters of the idea say that you're cheating if you use a ton of saves to make it through the game. Huh? Give the option of saving and then let people do whatever they want. Some people might n
Re: (Score:2)
I don't think I could stand the game without OOO. With it, it still gets
Re: (Score:2)
If you don't like Oblivion's Leveling system, there are lots of mods that let you change it. One of them lets you temporarily turn off your own leveling entirely so you can build up neglected skills without having it throw everything out of balance. Others like Oscuro's Oblivion Overhaul get rid of the leveled list monsters and loot (for the most part -- they still leave some of it in).
Playing on a console, not much I can do there.
I don't think I could stand the game without OOO. With it, it still gets .. a 9/10 (points off for a much shallower plot than Morrowind, poor voice acting, and overuse of bloom in outdoor areas)
Yup. That's certainly one of the advantages of playing on PC, you get all sorts of hacks. I'm at the part where I'm enlisting aid for Bruma. God, this sucks. There's gates outside of every frickin' city I go to. Even worse, there's a quest marker saying there's someone in Imperial City I need to talk to but nobody shows up on the map when I'm there. I hope nobody important got killed. I've read in the guides that there are some side quests that are no longer a
Re: (Score:3, Interesting)
You forgot three words at the end of this sentence of your comment. "Fucking 10/10, absolutely, in my opinion." I played Soul Caliber for Dreamcast a few times at a friend's place. I
Re: (Score:2)
Shouldn't it be obvious that it's in their opinion? I wouldn't take any reviewer to give an absolute score on anything
Re: (Score:2)
You forgot three words at the end of this sentence of your comment. "Fucking 10/10, absolutely, in my opinion." I played Soul Caliber for Dreamcast a few times at a friend's place. I don't like playing fighting games all that much -- I prefer role-playing games -- so I'd probably rate it as a 6 or 7 on a scale of 1 to 10. Now give me Final Fantasy Tactics, or Castlevania: Symphony of the Night -- in my opinion, those go to 11. Your mileage may vary.
That should go without saying. I don't like football games, never have and never will. It would take a hell of an exception to make me want to play such a game. So naturally I will weight any review I see concerning a football game in such fashion. It would have to be an exceptional football game for me to play it. You could say the same thing about movies -- nothing will get me to watch a Hugh Grant romantic comedy -- but there are crossovers. I have no interest in soccer and sports movies but Shaolin Soc
Re:Exactly... (Score:4, Interesting)
This list doesn't cover the generics, where you don't make Easy this difficult [youtube.com].
The benefit of assuming perfect and stripping away points for known flaw patterns is that you can properly assess how well games stand up to others. It can also allow ratings to be "depreciated" in the same way that other assets do as new flaws get discovered. The disadvantage is that you need to have a lot of experience reviewing and playing games to know and recognize flaw patterns.
Re: (Score:2)
Re: (Score:2)
The first time you sleep in the game after killing a civilian, you get approached while you're asleep by a representative of the assassin's guild who comes and makes you an offer to join. The problem is that in my game, the first time I slept after killing a civilian happened to be in a dungeon with a half-dozen traps between myself and the door. The NPC appeared, gave me my offer, then was immediately killed by traps be
Re: (Score:2)
I got bit by the scripting in Oblivion. Spoilers ahead for anybody who hasn't played it.
I can't even imagine the hell of debugging scripted game events. The QA department must be paying off karma for something awful. The worst part from the designer's perspective is that they have absolutely no idea what the player could be doing to screw things up. No matter how secure they make it, the player can always screw it up. When I played the original Betrayal at Krondor, I explored far, far away from where the current chapter was. I took food from a lockbox that should have remained there. A few ch
Re: (Score:2)
Re: (Score:2)
What needs to be done is to realize that game
Re: (Score:2)
His Halo 3 review [escapistmagazine.com] was honest, critical, and has a poignant comment or two on the very subject of broken game revi
Re: (Score:2)
Just what are you comparing Halo 3's singleplayer campaign to? Half life's? [Which is one of the best FPS's in the genre.] Quake? Unreal? [I don't think those two even had single player campaigns.] Doom? [You best be joking if you honestly think ANY of the Doom's had a better singleplayer than Halo 3.] Maybe FEAR?
Mind letting us know just where Halo went wrong and your ultimate singleplayer FPS went right? Obviously gian
Reviews are bought (Score:2)
In the early days the developers would simply bribe the writers and they'd write a review without even playing the game! That kind of practise has changed but not by much for some illustrations.
Re:Reviews are bought (Score:5, Interesting)
The disadvantages:
-You wouldn't see the review until after launch. (Probably a week for some games.)
-It doesn't seem to have a viable revenue model, unless someone knows a counterexample?
Re: (Score:2)
Great, now I've got that stuck in my head now. Jerk.
Does it matter? (Score:2)
Though in general the score is only a minimal part of a good game review as every gamer has different tastes and a good review is one that doesn't just
Re: (Score:2)
On a personal level, I'd have no qualms with someone giving, say, Casablanca or Raiders of the Lost Ark '5 of 5 stars,' (like a movie review column) because they're both epito
Re:Does it matter? (Score:4, Funny)
When game rags claim to put their blood and sweat into the magazine, I never knew they meant it so LITERALLY.
Review scores (Score:2)
ZZAP!64 (Score:2)
Whoops... I just checked, actually they used % as well, for the various parts and the overall.
Numerical sciores suck... (Score:2)
What's the difference again? (Score:2)
There are some phenominally crappy 2 star movies, and there are some that are underrated.
There isn't an actual criteria behind each star.
All "100" really means is that there are 100 possible stars... and everyone I know treats it that way.
That is, if one game got 70, one got 80, then we know both are rated at similar quality. And having played other games in that range, I have an ide
Movies do a lot of things different (Score:2)
More importantly is the author's contention that we're grading games as products, not art. How many movie reviewers give separate scores to the special effects, audio quality, and re-watchability of a movie? Su
Reading article...hust my brain (Score:2)
Having 4 stars could just as much represent perfection as a game that got a 10. It is just as arbitrary. It is really hard to take this guy seriously after that. I still read on, though, but I should have stopped. This article was of the form: 1) Put up out of context quote. 2) Rant about it in a way less intelligent than other people already have.
If this is all it takes to be
Wow... (Score:2)
Chris Mattern
A better way to review (Score:2)
1. various gaming facets as compared to other games in same genre, and why.
2. overall gameplay vs. all games this year -- game length, learning curve/complexity, etc and why.
3. Does it make the gamer consider entering this game in their all-time favorite games, and WHY!
Don't forget tons of real gameplay video and screenshots, since the commercials will just have gratuitous and irrelevant animations.
Dunno (Score:2)
Game reviews do work but only if you help (Score:2)
It's all opinion people, plain and simple.
For instance, RPGs: I hated FFVII but I enjoyed DQ8. Survival horror: I thought RE4 was needlessly frustrating (and yes, I have the Wii version too) but I really enjoyed Eternal Darkness. RTS: Starcraft and War2 get
Re: (Score:2)
Sounds like typical marking - add weighting? (Score:2)
One of the reasons most courses don't mark past 80% is that 80% is unattainable - the only way to attain it is to have a copy of the marking scheme. When a marking scheme only identifies the points necessary to make a "complete" answer, and not every possible answer (impractical) it is just as guilty as cheating in this respect.
This is why we have grade adjustment, or weighting. You adjust a scor
Re: (Score:2)
This is why we have grade adjustment, or weighting. You adjust a score based on the "current level" - a weighting compared to peers. Should Half-life be reviewed now, it would score badly in graphics, animation and possibly sound. Game reviewers should look to creating a standardised weighting system that has regular reviews.
I took a high level genetics course once. I checked my results online and on the final I got 14%. I was crushed. I thought I had failed it for sure. I went in and check the stats sheet and found the top mark was 20%, the lowest mark was 5% and my 14% gave me a 7/9 int he course. some professors are just sadistic.
Reviews have been fundamentally broken for years (Score:4, Insightful)
Problem is that these magazines are at the mercy at the games they review. They need to get exclusives, interviews, previews and adds to stay in the game. They are therefore very reluctant to give out bad scores to games from well known publishers.
Once upon a time there was a magazine (Amiga Power?) that did just this, said things as they were, and they found themselves cahoots by devs like Team 17, etc, for simply stating their actual opinions.
How about a survey: (Score:5, Funny)
On a scale of 1 to 10, how bad do you think the 10 point rating system is.
It's an opinion guys, get over it. (Score:2)
In my moderate-gamer life I can think of a few 100-score games but I don't get in a huff when other people come by and tell me that "American McGee's Alice blows" or "The Thief series is overrated".
Why do we have to have this eternal debate over whether a quantifiable
It's about the content, not the score (Score:2)
Logarithmic scale (Score:2)
If there is no such thing as a perfect game, when why the hell are you scoring out of 100? [...] Games are penalized through being divided by a sum that they can never possibly reach.
Here's what I think about scoring out of 100. Have you noticed how it's as easy to go from 0 to 80 as it is to go from 80 to 90? Here's the thing, scores are not linear, they are logarithmic, so if you want to report a score out of 100 into a linear score, f(90) might just be twice as much as f(80), and f(100) is infinity. Th
What I Think is Really Wrong (Score:2)
I don't think I am full of original ideas on this topic, but I have some thought that weren't yet in this thread when I started typing.
There is nothing wrong with game reviews being out of 100; changing that number to 4 or 5 or a 100000 has nothing to do with what's wrong with game reviews. Here's what I think is wrong:
Your way, my way, and why none of it matters (Score:2)
This has been covered on GFW radio (Score:2)
Specifically, the point is made that game reviewers needs to bring themselves up to the level of book or movie reviewers. Good movie critics don't recite feature lists and technical specifications, they talk about the ideas behind the art and how well they were realised. They talk about subjective things, with no claims to being an objective review.
I approve of this article (Score:2)
You're not grading a math test. (Score:2, Interesting)
The problem is that reviewers don't take into account of the reason why grade school has this stratified curve; It's the curve you get when students are graded based on the percent of quantifiable probl
I edited a games mag in the 1990s... (Score:2)
I was told in no uncertain terms by my bosses that my mag would be using percentage scores until the day it died, because "that's how the industry works." In other words, if a game didn't score at least 85% overall, you'd be on the publisher's bad books. And it's easier to for a PR guy (or your
Movie reviewers fixed this... (Score:2)
People want to give a absolute rating (4 stars, 99pts 'this game rockz!') as it gives them some self appointed absolute ruling, for which there is no unit of measurement. Is a 90% sports game better or worse than a 90% FPS from gamespot equate to the same 90% from ign?
Who needs reviews when we have demos (Score:2)
I used to get game magazines 5-10 years ago mainly for the demo discs and articles about upcoming games, but with the internet, game magazines are pret
5 Stars (Score:2)
Kotaku review (Score:2)
published by Kotaku
Graphics: 3/10. There are some pictures and graphics on the side, but for the most part it's just text one a screen. I could forgive it not being 3D, but they're not even using sprites! The in-game ads are also kind of lame, considering how little effort they put into the graphics.
Sound: 1/10. No sound to speak of, just the hum of my computer's fan, which is kind of annoying.
Gameplay: 5/10. There's not a lot to the gameplay. Basically, you submit
Re: (Score:2)
Average is good, but I'd like to see range or standard deviation up there as well. I'd take a game with all it's reviews in the 70s over a game with a couple of 90s and a couple of 50s.
Re: (Score:2)
It becomes very difficult to rate games in the same way as movies. A four star movie from 10 or 20 years ago is typically still a very good movie. A game that would have rated a 85% or a 90% 10 or 20 years ago would not hold up to the same scrutiny today. Love it or hate it, video games exist in a moving target of expectations, and until we see a general leveling off in terms of artistic and technical capabilities, I don't think you'll see the reviewing camps move to a more general rating system.
Similar to movies in their first 30-50 years of existence. In those years the initial movies were crude, broadly drawn and over all "terrible". But as time went on the tools to tell a story matured and there is a more even ranking now. Games are still int heir first 30 or so years so you have to consider it the same way. Metropolis was brilliant for it's time, very hokey right now. Super Mario was brilliant for it's time but pretty hokey right now.
Re: (Score:2)
Not having multiplayer isn't a big deal. In fact some games (like Stranglehold) it would be better to not include it (If it sucks, as a rule it should be left out). I agree that Bioshock shouldn't have been slammed for lack of multip
Re: (Score:2)
Super Mario Bros isn't anything to look at graphically, but I'm playing The Lost Levels [nintendo.com] more on my Wii than any of my Wii games...