ON RATINGS AND RATING SYSTEMS

Standard

5 stars4 stars3 stars2 stars1 star

 

Figured for my 100th post on ten years too late i’d do something special, and i suppose doing something special means writing another long winded essay of sorts. BTW, buckle up this is a long one…

 

In my imaginary world, where the millions of people who read my random thoughts on this blog or just ask me questions about my opinions (BA HA HA HA!!), one interesting question is:

“Why do you use a scoring system for your reviews?”

Well, you’re going to get your answer now; the idea for this article started back in February when i read Yahtzee Crowshaw’s extra punctuation article. In it he posits that scores are ways of readers to be lazy, and are fundamentally flawed because you can’t jam a numerical value on a work of art, as art and opinion of it is so subjective to the reviewer and that all nuance of the opinion would be lost if given a arbitrary number.

I’ve read other articles that use the exact same argument, and i don’t agree for a number of reasons. First i’m going to rebuke what i find most disagreeable in Yahtzee’s article and then i will explain why i use the rating system that i do despite its failings (yes it does have flaws).

 

THE WRONG

“My policy against scores is rooted in the fact that their meanings are so nebulous that they have no meaning at all, and they exist only for readers who aren’t willing to spare the time to do more than glance at a summary at the bottom of the page… if a reader isn’t willing to read through the whole critique, then they clearly aren’t there to seek guidance on a purchase decision or to enter a higher cultural discussion on a game; they probably just want a quick piece of ammunition to affirm a view they have already made, to mindlessly support or mindlessly condemn.”

 

To not see the meaning behind a review score is to not understand the simple question they seek to answer, that being: “did you like it?” and where your feelings about it lie on a spectrum. It is a summary, and functions like the last paragraph of his and many other reviews or opinions. However instead of using words, a scale of some type is chosen. I really don’t see how placing an overall impression about a piece of media in a general sense is meaningless, unless he just doesn’t understand the purpose and/or disagrees with its principle (which he clearly does). I will go more into depth with this later on as i explain how i chose my scoring system.

How the reader chooses to use the score that i give is ultimately up to them. Review scores can and are being abused by people who are just too lazy and unthoughtful to formulate an opinion of their own. But that’s not the score’s fault, again the score is just the summary of the opinion. My opinion is expressed before the summary is given, it’s why i put the score at the end of my reviews. Yahtzee doesn’t like the fact that his opinion might be ignored in favour of a summary and that’s his right, but again that doesn’t mean all scores are meaningless. A decision like that is as arbitrary a statement as his argument makes review scores out to be.

 

“All it takes is for a ‘Recommended’ or an ‘Essential’ to be given to one game that’s isn’t all that, and the title is meaningless. Eurogamer’s new award system sets a speed record by becoming meaningless before the end of the article announcing it, when a list of recent games is displayed with their new shiny prizes attached. Every single one of them was Recommended, bar the first, which was Essential (Sunless Sea for the record), the Recommended list including Grim Fandango Remastered, Captain Toad Treasure Tracker, and Assetto Corsa. Now, surely they can’t all be equally recommended without caveats. I like Grim Fandango but I don’t like car games much. BOOSH SYSTEM DESTROYED.”

 

No, Yahtzee the system isn’t destroyed. Your argument was when you just used the rating to ignore the opinion, something you said you didn’t like happening… All of those reviews had opinions attached to them, all of them. They also were all written by different people with different opinions, but he didn’t want to read them because he has already made up his mind that all scores are bad. Let the irony of that sink in.

A blanket statement like this ignores the scale chosen for the scores; a scale chosen purposefully to aid in the summary of a complex opinion. After all why is Grim Fandango Remastered a recommended title and not essential one? It’s because there are indeed caveats that stand in the way, which is precisely why they are explained in the review, and it is precisely why they added the recommended rating in addition to essential and avoid ones. It really is bizarre that he tries to black and white a scale that is chosen specifically to represent three shades.

 

“But y’know, perhaps it’s giving up too easily to say that scores can’t possibly work. We know that a numerical scoring system works perfectly well if you’re reviewing something entirely functional, like a big knife. You just write down how many things it can chop before it goes blunt. You can’t do the same thing with art, say by writing down the number of times you cried/laughed/threw up during a drama/a comedy/The Human Centipede, because emotional response to art is entirely subjective. But it is true that video games have both an emotional artistic aspect and a functional, technical one. You won’t be throwing up at all if the fucking thing doesn’t run or control properly.”

 

Before i go on i just want to make it clear that I don’t want this article to turn into some personal attack on Yahtzee, i quite like his opinions and agree on many of them. But he shows his ignorance and favoritism for his chosen art form with a statement like this. A statement like this is precisely why i don’t review music albums, paintings, or interpretive dance. As I might say something so ignorant of the subject matter it offends someone.

There is a world of difference between a custom made blade by the likes of Jay Fisher to the stuff stamped out by Gerber. A blade can be a work of art, and indeed some knives made by bladesmiths do indeed cross over to being art. The difference is the same as comparing a handmade solid oak book shelf to a glue and composite one you pick up at Ikea. Both will hold books and do so well, but you can’t say that their isn’t a tangible intrinsic difference between the two.

Granted he may not be able to see the validity of knives being a work of art. That’s great, i don’t consider most video games to be art (that must be really annoying to hear for some) as i’ve only ever played one game in my life that i thought ascended past being pure entertainment. That is because art and its enjoyment is indeed subjective, and i’m a believer that just about anything can become a work of art. However art can be judged, and it can be judged on a scale or spectrum. Whether you use words or symbols or numbers to summarise the opinion is entirely up to the person expressing it.

 

“A single score is meaningless because a game that operates on (and can be appreciated on) multiple different levels. So there we have our answer: all we have to do is break down exactly what those levels are and score them all separately…”

 

This will be the last thing i touch on in his article, as his proposed method is purposefully ridiculous and i’m not wasting my time (and yours) rebutting his sarcasm.

I honestly kind of agree with the point he makes here. Much like how cinematography and editing are crucial in shaping the experience of Cinema, how well you control a video game is crucial to the enjoyment of the experience. In fact control is what makes video games unique in all media and art; as it is the only one where the experience isn’t passive to the viewer.

I don’t score out all the aspects of film production for the purposes of brevity. After all the score is a summary of my opinion not my opinion in and of itself. I use the summary to catagorise the media i review, and as a recommendation of the overall worth of the piece. If a film has music, or editing, or cinematography worth noting and affected my views (good or bad) i will try and make note of it in my review, as it will effect the final score.

 

THE RATING

As i said before, the whole point of a score at the end of a review is to give an overall and indeed a nebulous summary of a piece of media. It answers the questions of “did you like it?”, “Do you recommend it?”, and “How much did you like it on a scale?” It’s the last question that i find most important to making a scoring system work. As a scoring system has to have an appropriate amount nuance and flexibility in order for it to fulfill its purpose.

As an example: i ask you the reader if you liked these films and take note of the first thing you say in your head:

RAN
Insomnia
Man of Steel
Star Trek V: The Final Frontier
Karas: The Prophecy

If i answer in a yes or no fashion it would be like this: Yes. Yes. No. No. No. With this i have the beginnings of a scale that can be used to score films. While some might find this adequate in summing up or expressing their opinion, i find it to be too restricting as indeed i agree with Yahtzee as he points out:

“…Now, surely they can’t all be equally recommended without caveats…”

Man of Steel, was the hardest for me to answer no to. That’s because i kind of like it, but it has some serious faults that prevent me from completely recommending it. I’m sure you can see where this is going; since i find yes or no to not satisfactorily sum up my final opinion of Man of Steel i have to adjust my scale to include “kind of” in order for the score to be valid. Now i can stop adjusting the scale with the inclusion of just those three ratings (just like Eurogamer has), but i find this scale to be too limiting for a number of reasons.

I would recommend watching Star Trek V over Karas any day of the week. ST: V is the better of the two in that it’s at the very least a coherent story. Thus i can increase the width of the scale by adding “Hell No” to it. The same can also be said for Insomnia and RAN, as i find RAN to be the superior film in terms of artistic breadth and merit. Thus i can add “Hell Yes” to the rating system. Here they are now in order:

RAN (hell yes)
Insomnia (yes)
Man of Steel (kind of)
Star Trek V: The Final Frontier (no)
Karas: The Prophecy (hell no)

Now i have the beginnings of a rating system that is nuanced enough to reflect my complex opinion of films. However it is still not complex enough for my purposes, i won’t elaborate further as i believe i have explained the process of growing the scale to a suitable end thoroughly enough that you as a reader can logically continue the process.

 

Before we move on, here is a very real problem that affects millions of people world wide.

 

 

THE PROBLEMS

The most logical rebuttal to my process would be the question of:

“Where do you stop? If the scale can be divided up infinitely then why don’t you just get rid of it entirely and express your opinion through words?”

Indeed if the scale constantly evolves it becomes useless as a summary and recommendation, as it will lead to confusion and defeat the purpose of what a nice neat summary should be. This is where i must concede that if a reviewer chooses to use a scoring system they must make an arbitrary choice to the “limits of the scale” and keep to them. Which i myself have done, in the form of the five star rating system.

I find the five star rating system to be complex enough to provide me with the appropriate amount of steps that sum up my opinion (as opposed to the four star version which doesn’t supply enough gradations to be satisfying), but still retain a simplicity that makes it easy to use (unlike pitchfork’s decimal system that supplies too much variation between gradients for me to consider useful).

Indeed i have run into the issue of my rating system not being complex enough to assign a rating to Apocalypse Now: Redux. As that film lies somewhere between a 2 1/2 and a 4 1/2 star rating for various listed reasons. With this in mind i can either find a new scale to account for this discrepancy, or i can continue to use the five star rating system that is the easiest and most suited to my needs nine times out of ten.

It also brings up the problem that it places disparate films like 13 Assassins and ST III: The Search for Spock  on the same footing as one another. This would be a problem if the score represented my opinion completely, but it doesn’t. My opinions on those two films are different however they leave the same impression on me and i can categorise and recommend them in the same manner (with a three star rating). This is once again because the rating functions as a summary and not the opinion in and of itself.

 

THE SUMMARY

This will inevitably lead to the question/rebuttal of:

“Well if you are making an arbitrary decision because the rating system fails in describing the opinion fully, aren’t you just confirming Yahtzee’s argument that scores are useless?”

My answer is: no. Opinions are decisions made on the worth of a topic or item, while they can be based on evidence they are inherently arbitrary as they are based on a subjective view point. It is a decision on my part to use a rating system, and i have chosen one that expresses my opinion in the most succinct way. Yahtzee has made the decision that all ratings are worthless, which i don’t agree with as i find ratings have value if used properly which i have argued in the above paragraphs.

Michael Mellody in this article approaches the problems with reviews with more tact than Yahtzee. But the common argument once again centers on how complex opinions can’t be expressed with an arbitrary score. He in his argument makes note that Pitchfork’s review system involves 101 different ratings, however he doesn’t acknowledge it is through those enormous amounts of gradations that it can function as a summary of the complex opinion of the review (which is usually attached to the rating via link or underneath). He instead sidesteps that, and focuses on how he feels one album is better than the other, ignoring that it is all just a matter of opinion in the first place!

Yes, ratings can hurt art and its growth when they are abused, but once again that isn’t the ratings fault; it’s how the people reading them use (or don’t use) the information given to them that hurts art and its appreciation. Yes, ratings seem arbitrary and stifling; until you factor in the attached review and see the rating as the summary it is meant to be. No, It doesn’t stifle debate with interested parties; as they will read the actual review or better yet, form an opinion of their own.

Please note, that i haven’t said that a review using a rating is superior to one that doesn’t use one, Yahtzee and Mellody say otherwise, and i don’t agree. What i mean to achieve with this post is to show the fallacy of a black and white argument on a matter of opinion. If you think that your opinion is too complex no matter how many gradations of scale you add to a rating system to be expressed as such, or you’re worried your voice will not be heard as people will TL:DR* your opinion you’re free to not use a rating system.

However, don’t think that yours is the superior methodology and that is especially so when a complex opinion proceeds or follows the rating. After all, the writer has given you the choice of looking at the review score and moving on, or looking at the review score and reading why it was given it. It’s your call, just like your opinion is and just like how mine is too.

 

 

*TL:DR functions exactly like a rating, however instead of stars or letters or numbers, it uses short phrases to sum up a complex opinion.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s