Dec 1, 2010

Review Scores: Why I Don't Them

If you've been following this blog at all, you might've noticed that I've starting trying to refine my review structure a bit. You probably would also have noticed that I never give things a score, and that I still haven't started doing this. I assure you, this is not an oversight, and the purpose of this short rant is to explain why.

In short, I feel like all giving a game a numerical score achieves is giving more opportunity for my opinion on a game to be misunderstood or misconstrued, for two main reasons. Firstly, numerical scores on reviews are completely arbitrary, and secondly, if there's a score on a review, that's all that people tend to remember about it, rather than what was actually said.

The biggest point is probably how arbitrary they really are and how much they're meaning differs from person to person. To me, a 7/10 is something that's quite good, and worthy of praise. A 4/10 is a little bit below average, but still a long way from being really terrible. Compare this to what you see on most mainstream gaming sites. A 7/10 is generally an average score, and anything less than 6 normally means that the game was very bad.

Generally speaking, if I read a review on IGN or Gamespot, the only thing I remember is the score. I don't tend to remember much of what was actually said about a game, I just remember X game was a 9.7 and Y game was a 7.4. Because these numbers don't actually mean anything concrete, this actually means that I still know effectively nothing about a game, and a game that was a 7 for someone else may well be a 9 for me, because the problems with the game that were very big issues for somebody else hardly bother me, for example, or because I like the style of game more and am a fan of the source material.

And who hasn't just skipped to the end of a review to check the score? That's really the most poisonous thing of all. It gives you the impression that you know something about a game, when in reality you know precisely nothing except for what number somebody who isn't you assigned to it. What I hope for is that, in lieu of a score, if people don't want to read my full reviews, they'll skip to the “Verdict” section, and read 1-3 short paragraphs that sum up my feelings on the game. Surely this is much more useful and informative than a number.

I've still considered adding scores though. Here's how I'd do it – scale of 1-10 with increments of 0.5 (because what the hell is the difference between 8.2 and 8.3 on a practical level anyway?) and the different scores would mean approximately this;

1 – Game was terrible, completely irredeemable, developers should be shot. If you like this game, I would probably try to argue that your opinion is objectively and provably incorrect.

2 – Very, very bad. Would be surprised if anybody likes it.

3 – Heavily flawed product that's only likely to be enjoyed by serious fans of the genre or source material who are able to overlook the mass of shortcomings.

4 – Below average title with more bad about it than good, but at least some redeeming features.

5 – Average game. Has some issues but was playable without being particularly enjoyable or particularly painful, and some aspects of the game made it a little worthwhile. Cavalcade of Mediocrity.

5.5 – Probably put Alpha Protocol here – maybe a 6. Keep in mind, I LIKED Alpha Protocol a fair bit.

6 – Above average. A fairly solid game with some pretty good parts but generally marred by being a little bit dull or derivative. Probably Uncharted: Drake's Fortune.

7 – Good. A good game which should be enjoyable to the majority of players, provided they normally go for games in that genre. A “7” RPG will probably be regarded as a pretty good game by people who like RPGs, or as a very average game by people who aren't really into them.

7.5 – Probably Uncharted 2. Again, this is a game that I enjoyed quite a lot, actually.

8 – Very good. A strong title which is a very good example of the genre, and may even appeal to players who don't typically delve into that style of game. Probably put Medal of Honor SP and Risen (MAYBE an 8.5) in this area.

9 – Exceptional. This describes a game that is so good that it should easily be recognised as an excellent title, even by those who don't normally appreciate its genre or style.

9.5 – Maybe Deus Ex?

10 – Flawless. I cannot think of a single way to improve this game. To play it is to achieve nirvana. Essentially unattainable.

Maybe if there's enough demand I'll go with this. Maybe. But I don't like the idea of people glancing at a review, seeing I gave something a 5 or a 6, and assuming it's not a good game. Because it very well could be. And I wouldn't want to spread misinformation, or misrepresent my own opinion, or something.

Besides, it's way more of an ego trip if I get to think that people actually read – and therefore at least care a little bit about – what I think.

3 comments:

  1. Don't. Sweet merciful Christ, don't. You pointed out in the first half of the rant that assigning scores and numbers to something as complex and personal -- highlight, underline, bold and italicise that -- opinion of a single individual is not indictative of the product's worth to someone else who has a considerably large number of their own variables to add into whether or not they'd enjoy it.

    A good example is Battlefield: Bad Company 2. I think its a fantastic game that, while it has shortcomings as all things do, overall is an excellent first-person shooter in a very refined package. On the other hand, your experience was somewhat marred by your system specs and you've not since been able to try it again, and you continue to maintain that it's not quite your thing.

    While journos and amateur writers like yourself try or are supposed to remain neutral, there is always personal opinions, experiences, system specs [for PC], flavours, and all sorts of things that will heavily impact the final score.

    The ideal way to decide if a game is for you is to play it, but not everyone has cash to throw about like candy. The next best thing is to read a series of [hopefully well-written] reviews to come to your own conclusion if you are unsure, and maybe factor in the opinions of friends if any have it.

    Food for thought. Also your site is advertising that I check out some singles website, which is pretty good/horribly stereotyped towards your blog's demographic.

    ReplyDelete
  2. New idea for new review scheme: Reviews that are actually completely objective.

    I think neutrality in journalism (unless it's the kind where you're just describing events - but I'm referring more to opinion pieces and similar, including reviews) is highly overrated anyway. There's no reason to try to be objective if you're writing a review because it all comes down to opinion anyway. A review that was actually fairly objective would also probably actually be fairly useless. "It's a game. It consists largely of shooting people. It has 3D graphics, and also sound effects and music." etc. As soon as you add any kind of judgment about quality (i.e, the thing that actually makes a review useful) it's not even remotely objective, as that is entirely opinion.

    And yeah, one of the problems with scoring systems is that they give the illusion of some kind of authoritative, objective 'ranking' of a product when of course each person's enjoyment is entirely dependent on them. Hell, even the way I describe the scoring system I might use falls into this trap.

    Besides, I kind of associate the whole scoring system thing as being part of the horrible cancer that makes up most of mainstream gaming journalism at the moment.

    ReplyDelete
  3. I agree that review scores will more than likely undermine the integrity of the review and how it is perceived. However, If you do decide to pursue review scores i strongly advise going down the 'less is more' path for the number of score 'tiers'. A rating out of 5 (only full number increments) with a clear guideline to how these should be interpreted, provides a relatively loose (but meaningful) core perception of the game that can orient the reader.

    ReplyDelete