Slashdot Log In
Game Reviews are Broken?
Posted by
Zonk
on Wed Oct 31, 2007 11:08 AM
from the plz-fix-k-thnks dept.
from the plz-fix-k-thnks dept.
Kotaku is running an opinion piece looking at the process of reviewing videogames, and comes to the conclusion that the whole system is entirely broken. Author Mark Wilson takes potshots at the concept of assigning a numerical valuation to a game, and the emphasis on product reviews rather than content reviews. "If there is no such thing as a perfect game, when why the hell are you scoring out of 100? It's not just PC Gamer that thinks this way--most publications, even those who do give out 'perfect' scores, do so begrudgingly. It's as if the developer has somehow cheated and broken their system. The movie reviewers solved this problem a long time ago. That's why most adopted a simpler rating system in which a 4-star movie didn't imply 'perfection' but supreme excellence. In most cases, games are penalized through being divided by a sum that they can never possibly reach."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Not New (Score:5, Insightful)
Game publishers, consumers, and even the reviewers themselves have been going on about the shortcomings of the current system for quite a while now. Yet we never see any alternatives being proposed. I say to the article writer, "Yes, I agree that the current system sucks. But what is your alternative?"
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
You see, there's no such thing as a perfect restaurant, either, but there -are- 5-star restaurants. That's 5/5 on the game scoring chart. Or 10/10. Or 100/100. It's all the same thing.
Just like a restaurant, 10/10 for a gaming score simply means 'it's as good as could possibly be expected' or 'it's better than expected, even when you expect excellence.' Nobody claims it means perfection anymore since t
Re: (Score:2)
Rating on a percentage (or equivalently, a 10-point scale with one decimal place) is really only good for wanking around at the top of the scale, making sure that X game has a higher rating than Y, which is better than Z (totally independent of the kegs, gadgets, or bags of money, right?). That's what gets tossed out with, for example, a five-star system. Is it really important to me whether
Re:Not New (Score:5, Interesting)
Am I the only one who finds that comment just odd? While I can agree with you that the system is broken because there's no such thing at a "zero" rated game, but I do not see the difference between 100 points and 4 stars, besides it being simply "divided by 25". Then, of course, that's not good enough, so they start assigning 0.5 stars.
Also, I never considered "100/100" to be perfection, but as "supreme excellence" as noted. After, WTF is the difference between "supreme excellence" and "perfection"? Someone is just trying to argue semantics.
Of course, I don't even like the "four star" or 100 point numerical system. When I ask/tell people about a movie I simply say "Is it worth seeing in the theater?", "Is it worth a theater matinée?", "Is it a rental?" or "not worth your time, period".
In this sense, I saw the "Number 23" in the theater and I recommended that it was worth seeing in the theater. While "28 Weeks Later" was easily worth waiting for a rental (despite being a fan of "28 Days Later" and zombie films in general).
In that regards, I would say games should be rated as "buy it!", "rent it/demo it!", "stay away". (rent for console / Demo for PC games). Guitar Hero games are "buy it" games while something like Zelda:Twilight Princesses might be a "rent it" kind of game (I bought it, I'm a fan of Zelda, but still feel money better spent on a rental. I would not have given it 100/100 as some reviews did). World of Warcraft? "Buy it".
Of course, such a system needs a context. I'm not going to tell a FPS fan to buy an RPG, it's in the context of RPG fans.
Cheers,
Fozzy
Parent
Re: (Score:2)
If you spend time just enjoying the ambience of that game then it would presumably cost you far more to rent it? I spent a few hours one night just fishing.. definitely a 'buy it'..
Maybe (Score:5, Informative)
Although he has a point, we humans love to compare and if you don't give us any metric by which to do that, then we don't feel like anything has been achieved.
"SuperGame is really good" is meaningless to me. What I want to know is, is it any better than GreatGame? If the reviewer gives a score for both then I can understand which he/she feels is better and by what margin. Since I've played GreatGame (and assuming I trust the reviewer), then I can set some sort of expectation of what SuperGame will be like.
Personally, I use Metacritic [metacritic.com] which aggregates a number of reviews. Again, it's not perfect, but when it gets a 75 or above score, I can be reasonably certain that I'm not getting a dud game. It might not be my type of game, but if it is, then it shouldn't be disappointing.
Re: (Score:3, Insightful)
Re: (Score:2)
Except that the scores aren't derived from any kind of objective framework, so comparing the scores of two games really isn't meaningful. Maybe the reviewer's wife isn't giving him any that week and he's ticked off, so all his scores are 10% lower. Maybe his favorite American Idol contestant won that week and all his scores are 7% higher. The scores aren't scientific; they're completely subjective
Re: (Score:2)
Re: (Score:3, Interesting)
Yup, I find metacritic quite useful (I think gamerankings is another site among those lines), however I think that a service similar to what Criticker [criticker.com] provides for films would be good. In criticker you put a score to serveral movies you h
The best review (Score:5, Insightful)
The best review has no score. Simply somebody playing the dang game, and talking about what they like, what they don't like, what they'd improve, what really bothered them, what really excited them.
Find a reviewer with a decent command of the language, and who likes the sorts of games you like, and you're good to go.
Re:The best review (Score:5, Informative)
This is why I like the Zero Punctuation [escapistmagazine.com] reviews so much. Yahtzee has a decent command of the language, goes through all of the good and bad parts of the games, and gives a quick conclusion stating his opinion of the thing.
Parent
Re:The best review (Score:5, Insightful)
As TFA states, the review industry is necessary because of the large amount of games coming out and the large proportion of crap that inhabits it. If something is crap from end to end and at least four different people agree, then there's no need for me to look further.
Parent
Re: (Score:2)
The problem I have with that is that around number 2000 or so, the reviewer is knocking points off for 'there's nothing here that we havne't seen before.'
I don't care that the game is very similar to the other several RTSes, say, that came out that month; I'm interested in that game as that game.
A good example (Score:2)
Re: (Score:2)
Exactly... (Score:5, Insightful)
Re: (Score:2)
This is sort of my take. Halo 3 is a definite example. I know it's supposed to be good, but it has some real flaws that should prevent it from getting a perfect score. Many magazines and such seem to operate on the Famitsu model (as I had it explained to me). Some games just get high scores because they are fun an are expected to (say FF: XIII will). The more obscure games get real ratings.
At this point, while I read other reviews (and use sites likes Game Rankings), I really like X-Play. They tend to do a
Re: (Score:2)
The unfortunate thing is that many sites/mags aren't trustworthy. It's one thing to give Halo 3 a 5/5 when you point out some flaws and weaknesses but say that the game is fun and a real blast in multiplayer. But I've seen many reviews of games (other games, I haven't read much on Halo 3) where reviews just pass over that kind of stuff because it seems that not what people want to hear.
I agree with Halo 3. It had very poor graphic sin the cut scenes. It was visually blah, fun single player but short, and multiplayer is fun but not all that new. If I was a game review site it would be a solid 3/5 or 8/10. Fun, short, good multi-player, and remarkable mostly for it's hype and sales.
Re: (Score:3, Insightful)
Re: (Score:2)
These very thoughts came to my mind when I was reading all the Halo3 Reviews... When a game has so-so single player and awesome multiplayer...how does that get the game loads of perfect scores? A perfect game wouldn't need to make up for areas of lacking ANYWHERE. That aside, even the multiplay, while fun, is far from perfect. Halo3 was, and is, a great package but it's nowhere near a "perfect" game. I'm not just picking on Halo3 here either. HL2 for instance was a phenomenal game....but to call it "perfect" (like so many reviewers did) is just naive and downright inaccurate.
The last perfect score I remember seeing was for Soul Caliber on the Dreamcast. Gotta say, it is perfect. The only possible quibble I can come up with is that they could have had maybe a dozen voice samples for the announcer to cycle through to avoid repetition. That's about all I can fault it on! Fucking 10/10, absolutely.
I'm playing through Oblivion right now. That game is easily, easily a 9/10. It really lives up to the hype and deserves to be one of the top-rated games on console and PC. However, there
Re: (Score:2)
I'm playing through Oblivion right now. That game is easily, easily a 9/10. It really lives up to the hype and deserves to be one of the top-rated games on console and PC.
And that demonstrates the uselessness of numeric-only scores. I'd give it a 4/10, max.
That doesn't tell anyone anything other than you liked the game and I didn't. My 4/10 doesn't relate that I thought the game was a $50 GeForce stress-test, only slightly less entertaining than clipping my toenails; that the leveling system was stupidly broken, the storyline a snoozer, and the "openness" of the world including beast level-scaling was more a bug than a feature since it contributed to the stupid brokenness
Re: (Score:2)
I don't think I could stand the game without OOO. With it, it still gets
Re: (Score:3, Interesting)
You forgot three words at the end of this sentence of your comment. "Fucking 10/10, absolutely, in my opinion." I played Soul Caliber for Dreamcast a few times at a friend's place. I
Re:Exactly... (Score:4, Interesting)
This list doesn't cover the generics, where you don't make Easy this difficult [youtube.com].
The benefit of assuming perfect and stripping away points for known flaw patterns is that you can properly assess how well games stand up to others. It can also allow ratings to be "depreciated" in the same way that other assets do as new flaws get discovered. The disadvantage is that you need to have a lot of experience reviewing and playing games to know and recognize flaw patterns.
Parent
Re: (Score:2)
Re: (Score:2)
What needs to be done is to realize that game
Reviews are bought (Score:2)
In the early days the developers would simply bribe the writers and they'd write a review without even playing the game! That kind of practise has changed but not by much for some illustrations.
Re:Reviews are bought (Score:5, Interesting)
The disadvantages:
-You wouldn't see the review until after launch. (Probably a week for some games.)
-It doesn't seem to have a viable revenue model, unless someone knows a counterexample?
Parent
Re: (Score:2)
Great, now I've got that stuck in my head now. Jerk.
Does it matter? (Score:2)
Though in general the score is only a minimal part of a good game review as every gamer has different tastes and a good review is one that doesn't just
Re: (Score:2)
On a personal level, I'd have no qualms with someone giving, say, Casablanca or Raiders of the Lost Ark '5 of 5 stars,' (like a movie review column) because they're both epito
Re:Does it matter? (Score:4, Funny)
When game rags claim to put their blood and sweat into the magazine, I never knew they meant it so LITERALLY.
Parent
Review scores (Score:2)
ZZAP!64 (Score:2)
Whoops... I just checked, actually they used % as well, for the various parts and the overall.
Numerical sciores suck... (Score:2)
What's the difference again? (Score:2)
There are some phenominally crappy 2 star movies, and there are some that are underrated.
There isn't an actual criteria behind each star.
All "100" really means is that there are 100 possible stars... and everyone I know treats it that way.
That is, if one game got 70, one got 80, then we know both are rated at similar quality. And having played other games in that range, I have an ide
Reading article...hust my brain (Score:2)
Having 4 stars could just as much represent perfection as a game that got a 10. It is just as arbitrary. It is really hard to take this guy seriously after that. I still read on, though, but I should have stopped. This article was of the form: 1) Put up out of context quote. 2) Rant about it in a way less intelligent than other people already have.
If this is all it takes to be
Wow... (Score:2)
Chris Mattern
A better way to review (Score:2)
1. various gaming facets as compared to other games in same genre, and why.
2. overall gameplay vs. all games this year -- game length, learning curve/complexity, etc and why.
3. Does it make the gamer consider entering this game in their all-time favorite games, and WHY!
Don't forget tons of real gameplay video and screenshots, since the commercials will just have gratuitous and irrelevant animations.
Dunno (Score:2)
Game reviews do work but only if you help (Score:2)
It's all opinion people, plain and simple.
For instance, RPGs: I hated FFVII but I enjoyed DQ8. Survival horror: I thought RE4 was needlessly frustrating (and yes, I have the Wii version too) but I really enjoyed Eternal Darkness. RTS: Starcraft and War2 get
Sounds like typical marking - add weighting? (Score:2)
One of the reasons most courses don't mark past 80% is that 80% is unattainable - the only way to attain it is to have a copy of the marking scheme. When a marking scheme only identifies the points necessary to make a "complete" answer, and not every possible answer (impractical) it is just as guilty as cheating in this respect.
This is why we have grade adjustment, or weighting. You adjust a scor
Re: (Score:2)
This is why we have grade adjustment, or weighting. You adjust a score based on the "current level" - a weighting compared to peers. Should Half-life be reviewed now, it would score badly in graphics, animation and possibly sound. Game reviewers should look to creating a standardised weighting system that has regular reviews.
I took a high level genetics course once. I checked my results online and on the final I got 14%. I was crushed. I thought I had failed it for sure. I went in and check the stats sheet and found the top mark was 20%, the lowest mark was 5% and my 14% gave me a 7/9 int he course. some professors are just sadistic.
Reviews have been fundamentally broken for years (Score:4, Insightful)
Problem is that these magazines are at the mercy at the games they review. They need to get exclusives, interviews, previews and adds to stay in the game. They are therefore very reluctant to give out bad scores to games from well known publishers.
Once upon a time there was a magazine (Amiga Power?) that did just this, said things as they were, and they found themselves cahoots by devs like Team 17, etc, for simply stating their actual opinions.
How about a survey: (Score:5, Funny)
On a scale of 1 to 10, how bad do you think the 10 point rating system is.
It's an opinion guys, get over it. (Score:2)
In my moderate-gamer life I can think of a few 100-score games but I don't get in a huff when other people come by and tell me that "American McGee's Alice blows" or "The Thief series is overrated".
Why do we have to have this eternal debate over whether a quantifiable
It's about the content, not the score (Score:2)
Re: (Score:2)
Average is good, but I'd like to see range or standard deviation up there as well. I'd take a game with all it's reviews in the 70s over a game with a couple of 90s and a couple of 50s.