On reviews, scores, and objectivity vs. subjectivity

I’m in despair again.  This time about review scores.

It’s never not a good time to use screenshots from SZS

Let me back up about a decade and a half (I promise there’s a point to this trip through time, so don’t worry.)  Back in my school days, I used to follow two music reviewers: George Starostin and Mark Prindle.  These guys maintained websites dedicated to writing album reviews well before the modern age of easy blogging — before technologically untalented people like me could start free WordPress and Blogger accounts and dump words onto the internet without knowing anything beyond the most basic HTML tags.  Messrs. Starostin and Prindle were both excellent writers, very knowledgeable about music, and incredibly prolific (in fact, Starostin is still writing at a different address, though he seems to be on hiatus right now.)  Most importantly to me, they were independent voices that I felt I could trust far more than the hacks at Rolling Stone, Spin, and the other big music magazines.

However, Starostin and Prindle’s review styles were very different.  Starostin seemed to try to take a more objective approach to his music reviews.  While admitting that he couldn’t be totally objective, being a human with his own likes and dislikes when it came to music, he still tried giving a fair chance to artists whose styles he wasn’t naturally fond of (though he could and would tear an album up in a very entertaining way if he thought it was lousy.)  Prindle, by contrast, seemed not to give a damn about even trying to be objective.  He could and often did also write deep and interesting analyses of albums, but they also felt more personal in the sense that you were getting his opinions based purely on what he liked and disliked.  Prindle’s more personal style also came out in the various rants, anecdotes, and obscene jokes he’d drop into his reviews, usually without any warning to the reader.  Even though their styles were so different, I liked them pretty much equally, and I’m sure both of them have had a serious influence on my own reviewing style.

Source.  Though how “Movie X no longer has a 100% RT score” could be considered a story worth writing about, I have no fucking clue.

Now back to the present day, where people on Twitter and other platforms are tearing their hair out over the Rotten Tomatoes scores movies get.  Red Metal at Extra Life covered this already in a recent post about the reaction to the film Lady Bird getting one bad review from a critic, knocking its score down from 100% to 99%.  Some people were apparently losing their shit over this development.  If it can even be called a “development”, really.  No doubt they’d also be piling onto Red Metal if his own mixed review of Lady Bird had been factored into that score.  I haven’t seen the film, but I can say at the very least it’s impressive that a movie managed to get such dedicated fans that they’d scream bloody murder over a single poor review.

Or is that really what’s going on?  It looks to me like many people have expectations that certain artworks should be insulated from negative criticism, as though they have a God-given right to a perfect score on RT and maybe also on every other review score aggregator.  I have no idea where these expectations come from.  Even among my favorite games and albums, I can’t think of a single one that I’d yell at a reviewer for over a poor review.  I’d certainly disagree with said review, but as long as it was reasoned out well enough, I’d just think “Fine, that person has a different opinion than I do.”  Because we all have different tastes, different perspectives, different life experiences.  Not everyone has to like what I like, and I don’t have to like something even if almost everyone else likes it.

I like drinking beer, chewing on dried squid, and playing visual novels, but a lot of people don’t, and that’s okay.

So how should I approach my own reviews?  I’ve been writing reviews of games and other media for six years now (not on a very regular basis, as you can tell from looking at my index of reviews and dividing their number into six, but still, six years is a long time.)  I always try to write my reviews in such a way that they’re useful to every reader who follows this site or comes across it through a Google search.  But when it comes to the score I assign a work, I sometimes find myself facing this conundrum: if I score the work based too much upon my own subjective tastes, the score won’t be meaningful to a reader with different tastes from my own, and if I score it based too much upon some kind of as-objective-as-possible balance of factors, I’m removing my own views from the process so completely that I may as well not review the work at all.

I usually try to strike a balance between these two extremes, but sometimes that’s difficult, especially when the work I’m analyzing is directed at a niche audience.  I’m facing just this issue with the game review I’m currently writing.  Maybe I should just not worry about the problem at all and write whatever I want like Prindle, or maybe I should still try to take a more objective view of things like Starostin.  Maybe I’m overthinking this like I overthink every single other aspect of my fucking life.

Maybe don’t worry about cutting the cake precisely Chiri, maybe just cut it and eat some god damn cake

I have another question for you, the reader: if you write reviews, do you run into this problem?  How do you resolve it?  Or is it even really a problem and am I just overthinking things? If you don’t write reviews but only read them, do you really care about how objective or subjective the reviewer is trying to be?  And should anyone even care about Rotten Tomatoes or Metacritic scores aside from the film and game studios and distributors?

Sorry, that was more than one question.  You don’t have to answer all of them if you don’t feel like it.  Or any of them.  In the meantime, I’ll go back to finishing my next review.  Maybe one day, I’ll write a review that will get me a headline on Indiewire about how I’m an asshole who made people on Twitter cry.  I can only hope.

Advertisements

18 thoughts on “On reviews, scores, and objectivity vs. subjectivity

  1. That’s the thing about reviews: everyone is entitled to their own opinion of a piece of subject matter and no matter how you try to slice it, it is just an opinion. I use to claim to be clearly objective but realized that is not completely true since I’m just going through a specific work and pointing out what it is and why it works or doesn’t work to me. Sometimes, I do want to be a little more – involved in the process like joking around or trying to tear down and examine certain aspects. It would be a very boring review otherwise, especially for something I might not have liked or feel sparse in content, but still want to evaluate it. It all comes down to personal style and how you like to consume content. There is no specific right or wrong way to do it in my own opinion.

    • You’re right that we’re all just writing about our opinions in the end. There are technical details of works that you could say are objectively good or bad, but we all experience the end products differently if they’re at least competently made. I guess a truly objective review of a work like that isn’t even possible to write.

      • I would like to think it possible to be completely objective, but yeah, I don’t really believe someone can be. There is always going a little piece of you somewhere in the review. Nothing wrong with that. One reason I don’t like reading reviews from more “professional” critics is that they sometimes overstep the boundary and sometimes color other people’s perspective that might be willing to try something, but the review is so effective it does the opposite. That and specific sites don’t bother to understand the content they are reviewing.

        Out all the review sites, places like DigitallyDownloaded was a big influence on me. It is one of the reasons I decided not to do the whole “5/5” or whatever rating system. Even though my way of doing things leads to ambiguity it is a little more concrete than trying to assign something an arbitrary value without being precise on the criteria. Although, I respect and admire the that people can.

      • It annoys me when those pro critics write about games and totally mischaracterize them or don’t bother to do any research. It’s reckless.

        I can understand not wanting to use a rating system either. I didn’t use one for a long time. Since adopting this seven-point system I’m using now, it’s mostly worked out for me, but sometimes I have to really think about what score to apply to a game or album or whatever I’m reviewing. I’ve already bent my own system a few times in the process by using fractions. I’m the sole admin/writer here, so I guess it doesn’t matter too much.

  2. I feel like you should approach your reviews in whatever way you find more comfortable. You’re the one doing the review so people shouldn’t alter the way you do it. Trying to fit into a mold will just mean you’ll be like someone else. There obviously are going to be similarities between reviewers but there will always be something that makes them stand out.

    Reviewing style comes from experience. Just like drawing. At first, they are whatever. They don’t have a particular style and just fluctuate between styles. Eventually, you’ll naturally develop a writing style. Since you have made reviews for 6 years now you probably developed a style.

    When it comes to giving ratings, I believe that you need to be more objective regardless of how subjective your reviews are. I liked plenty of games which are objectively bad but I won’t give them a 8/10 or 9/10. You need to provide facts as to why the game is good but also mention it’s flaws. The score is given by how much the flaws take away from the good parts of the game. This can’t be entirely objective, but everyone can acknowledge flaws.

    • Yeah, I don’t intend to change my writing style at all, as long-winded as it is (I don’t think I even could at this point if I tried) or probably even my reviewing style. When I think of the music reviewers I used to follow, I wouldn’t have liked it if they’d changed their own styles to conform to someone else’s expectations.

      That’s a good point about enjoying games that are bad, especially if they have technical problems or issues with story structure or characters that anyone with a critical eye would be able to point to. I tend to get frustrated with games like that, but there are quite a few really bad movies that I like (like say The Happening, or a few of the old really terrible sci-fi movies MST3K used to cover.) I still wouldn’t give those movies high scores just because I enjoy them on some level. In any case, as long as we’re honest and thorough with our reviews, we’re doing right by our readers, whether they agree with our assessments or not.

  3. For me personally, entirely subjective. My reviews are my opinion based on my experience with a game. The thing is, I also try to include examples for why I think something is or isn’t good/bad. I do that so that when someone who doesn’t agree with my opinion comes along and reads it they can go, “Ah…I see why you have that opinion.” I treat reviews like a summary of honest statements based on my time with the game. How successful I am at that…well that’s up for debate.

    The other thing I don’t do is add scores. I find that far too many people want information to be nice and simple to consume. Because of that, they’ll skip over reading the entire review and just argue about the score a game gets. So I avoid letting anyone have that opportunity with my stuff. :p

    As for what you should do: that’s entirely down to personal preference. I knew going into reviews that I wasn’t capable of writing anything that could be considered objective, so I didn’t bother. For me the choice was easy. As you said, it’s about determining what kind of a balance you want to strike in your reviews between objectivity and subjectivity.

    Either way, looking forward to whatever your next review is.

    • I like that approach — an honest assessment of our experiences with these games are the best way to review them. As far as the use of scores goes, I get what you’re saying. There probably is too much attention given to scores and ratings (just look at the Rotten Tomatoes-related nonsense going on even today.) My rating system is non-intuitive enough that I hope readers wouldn’t just skip down to see the score, but I have no idea if that actually works.

      And thanks. Unless I end up pushing it back, my next review is going to be of a game series that a lot of people are probably embarrassed to admit or won’t admit that they play. Which just makes it more fun for me to write about.

  4. I think the most likely outcome is that these journalists wouldn’t take my opinion of Lady Bird seriously and leave my site alone. On the other hand, if they went nuclear and sent their followers after me, I’d get free publicity, so I win either way.

    The Lady Bird incident is what happens when your critical circle has zero conviction in their own opinions. It’s interesting because for the longest time, I thought maybe it was just games or other works with cult fanbases that exhibited behavior like this due to being far away from the reality in terms of themes, yet A24, a company that often prides itself in realism, has a fanbase worse than any gaming one I’ve ever encountered. As bad as fans of, say, Portal, Sonic, or Undertale get, at least you get the sense that the people with actual weight behind their words are embarrassed by them. A24 fans, on the other hand, actively encourage this toxic behavior – and the reason it rarely gets challenged is because A) film fans are much more fractured than gamers and B) the professionals join in on the fun – in many cases (such as Lady Bird) they even instigate these incidents. Plus, as bad as those fandoms can get, at least I get the sense that they enjoy what they obsess over; I don’t think film critics (or even cinephiles) can claim the same.

    It’s debatable as to whether or not A24 has changed the art of filmmaking for the better, but I can say with 100% certainty that they have been a horrible influence on critics/journalists. I still consider how they handled the Ex Machina viral marketing campaign (another A24 production, incidentally) a “jump the shark” moment for professional film journalism. They had always, to some degree, cared more about their art than their audience or those potentially harmed by an artist they put on a pedestal, but with that gesture, they threw their readers under a bus without even the slightest bit of hesitation. And then Lady Bird came along and demonstrated they don’t have any loyalty to each other; you’d have a better chance of finding honor among thieves. It’s a total mess, and going on the way they are, film critics are going to risk irrelevancy in the long term – and they’ll have no one to blame but themselves. Indeed, I see the stuff Owen Gleiberman and people like him write as a symptom of this inevitability.

    Honestly, I think we need more critics like Mark Prindle. Now, there was a guy who didn’t give a care as to what others thought of his opinion and was absolutely genuine about everything he ever wrote. I could see snobs dismissing him as a degenerate given his propensity for vulgar humor, but he was a very witty fellow who had actual passion about his craft – a far cry from the interchangeable cogs passing themselves off as film critics these days. I discovered a lot of interesting albums through reading his website (such as The Cows’ Cunning Stunts by and Slint’s Spiderland), and I have to say he was an influence on my own scoring system. You know how I have a rule stating that I can only award one 10/10 per franchise (or one 10/10 per director)? His site is where I got the idea from.

    Finally, to actually answer your question, I have to say that my style has changed quite a bit over the years. At first, I would actually use the Intro/Game/Story/Conclusion template – even if the story wasn’t important. Eventually, I learned to be a little more dynamic with the sections, though I would still lay out everything objective about it before giving my opinion on them. Now, I’ve become less rigid about how I write these reviews with objective facts dominating the introduction, subjective opinions dominating the conclusion, and a mixture of the two comprising the main body of my review.

    • I don’t know much about film fanbases vs. game fanbases, but it’s frightening to think that the A24 crowd is worse than the rabid Portal, Sonic, and Undertale fans. Would make sense if, as you say, their worst tendencies are enabled by prominent critics. I definitely get that sense from the writings of Mr. Gleiberman and others like him that there’s an “us vs. them” mentality, where those critics lead a circle of truly cultured people, and the rest of us unwashed masses are too dumb to understand their tastes. As many problems as I have with professional game critics, it’s true that they aren’t really trying to do that, or at least they aren’t yet. I think a lot of pro game critics and review sites are also sliding toward irrelevancy in their own way, but we’ll see where that goes in the next few years.

      I didn’t know you got that “one 10/10 per artist” idea from Prindle, but I remember he did that. You could really feel his strong feelings about music from how he wrote his reviews. I never felt like he was holding anything back, and I try to bring that kind of sincerity to my own reviews. Because that was one of the reasons I followed his site — you could always trust that he meant exactly what he said and that he had the knowledge to back it up.

      Yeah, I’ve noticed that general structure in your reviews; it’s very effective. I don’t really plan anything out beforehand; I just start writing until I’ve written a review and then edit it until I feel like I wouldn’t be embarrassed to post it. Don’t know if that always yields the best results, but I guess it’s worked out until now. My reviews have gotten a lot longer over the last year, though. I look back at some of my old reviews and feel like they’re inadequate, but I just have to keep moving forward.

      • I will say A24 fans are probably not as blatantly obnoxious as those three aforementioned fanbases, but the fact that their fanboyism/fangirlism is encouraged on a professional level makes them worse overall because there’s no restraint. These are the kinds of people who could witness many Tinder users get catfished as part of a viral marketing campaign and see nothing wrong with it as long as it promotes the art they love. And yes, if film critics reach irrelevancy, they’ll do so in the exact opposite way as gaming critics. Whereas gaming critics are too controlled by publishers, film critics desperately need reasonable boundaries. Otherwise, they do indeed promote that toxic “us vs. them” mentality. Still, for what it’s worth, I do trust gaming critics more if for no other reason than because the fans can reel them in if they step out of line (whether they like it or not).

        Yup, Mark Prindle is one half of the equation. The other half is another critic by the name of Piero Scaruffi. Admittedly, I kind of think of him as the prototypical hipster in how he is completely incapable of accepting that popular works can be good. In fact, in one case, he flat-out lied to resolve this disconnect, claiming John Coltrane wasn’t popular in his day when A Love Supreme ended up selling 500,000+ copies by 1970. However, unlike Mr. Gleiberman, I did learn of a lot of interesting works though pursuing his writings (Captain Beefheart’s Trout Mask Replica, Morphine’s Good, and The Pop Group’s Y among others). His grading scale is notoriously strict, and I decided I liked it because one really does have to work hard to obtain a 10/10; one shouldn’t hand them out like penny candy (though unlike him, I actually have awarded a few 10/10s). He’s where I got the “10 and 9 = masterpiece, 8 = get immediately, 7 = get eventually, and 6 = get if you’re a fan” aspect from.

        And thanks! That means a lot. It’s also worth noting that I actually start by writing the pros/cons list. For you, it’s a summation for my review, but for me, it’s a helpful outline that reminds myself of my talking points. And I’d say as long as you can make your style work, go right ahead. Some of the films considered the greatest of all time were made up on the fly, after all.

  5. This whole discussion is why I 1) don’t refer to the things I write as “reviews” (despite other people perpetually doing so, because apparently reviews are the only type of article about video games that exists) and 2) by extension, I don’t assign scores to them.

    For me, scoring something has always felt a bit weird. How do you measure it? How do you quantify it? “This game is 96% good”. What does that mean? Ultimately, not that much beyond “the person who wrote about this really liked it”. And I’ve never really felt you need a number for that; in fact, as we’ve seen on numerous occasions over the years, having a number at the end often discourages people from actually reading the arguments made and instead providing a kneejerk response — particularly given the skewed scale video game review in particular tend to operate on, where a 7/10 is borderline “bad” and only things with an 8 or more are worth even considering.

    Back in… 2011, I think, I was recommended the original Hyperdimension Neptunia by someone I chatted to online. She knew that I enjoyed Japanese games and was interested in finding out more about what modern anime-style games had to offer. I took her recommendation. I played the game without looking up any information on it. I adored it. (It subsequently became one of my all-time favourite series.) I wrote about it. *Then* I looked up the Metascore and reviews out of curiosity. 45%.

    It was that specific moment that I knew review scores no longer had any value to me — or any power over me.

    For me, I’d much rather read a detailed, well-argued and respectful analysis of a piece incorporating the things that the writer liked or disliked about it — the things that make *their* view of the work unique — than look at an arbitrarily assigned score. And so that’s what I, personally, strive to create.

    This is obviously my personal response to all this and I’m not saying it’s “right” for everyone; some people crave the sort of order and organisation that neatly quantifying things brings, and that’s fine. But my mind is far too chaotic for that; I want to hear juicy details.

    • People certainly obsess too much over scores. It makes sense that you’d bring up a game like Hyperdimension Neptunia — I believe these score aggregators are really extra-meaningless when it comes to games aimed at one particular audience or a hardcore fanbase. I had the same issue recently with Our World Is Ended. It got Metacritic scores in the 40s and 50s, whereas I would have graded it at least an 80, maybe an 85 if I’d been using that scale. That game simply wasn’t made for or directed at a wide audience. (That’s not even bringing up the several hit pieces and calls for censorship and all that bullshit.)

      I understand why you wouldn’t want to use a rating system. I try to write thorough, analytical reviews of games, so the scores I assign them might just be afterthoughts. I will agree that the scales most of the big sites use are weirdly skewed. Just as with those RT movie percentages, I’ve seen people get pissed off over their favorite games not getting at least a 9 or 9.5 out of 10, as if even an 8 is a bad score (this maybe applies more to “sacred cow” series like Final Fantasy.) As for my own scale that I stole from the IB program, a 4 out of 7 is a passing grade, which seems logical enough to me. That’s the score I got on my Math Standard Level exam, and I damn well got that credit.

      You also remind me that I should also keep writing non-review pieces on games. I might dig through the Megami Tensei series more — there’s a lot buried in there.

      • I’d be interested to read some MegaTen stuff. It’s not a series I know well at all outside of Persona, but I know there’s an absolute *ton* of interesting stuff to talk about. Go for it!

    • Yeah, I’d have to agree with Pete. In fact, I came down here to say much the same thing, but Pete already did it, so I’m just going to piggyback.

      All this stuff is inherently subjective. And the boldest, riskiest works are often going to end up with mixed reviews in amalgam, because some people score them highly because the unique features appeal to their individual tastes while others do just the opposite. You could look at it in amalgam, but that’s not going to give you an accurate picture of what it means to you if it appeals to your niche. Hell, there’s nobody’s tastes that you’re going to match perfectly. And to be frank, it’s impossible to look at an 8/10 on a review of something and determine what that something is and what it has to offer you.

      Except for in cases like what Red Metal does where there’s a clear feeling about a work each score represents, I don’t place stock in scores at all. Hence why I don’t use them. I like the meat of the review, the actual content therein, and I’ll go based on that to determine if something interests me or not. I know my tastes better than these reviewers do, and that helps me translate it much better. No matter how good the game is otherwise, I know I will hate it if it doesn’t control well, for example, while I’ll often go for poorly reviewed works that seem to have a unique and interesting idea at their core. And I’m happy that way. I don’t know that there’s much I’ve missed out on by ignoring the actual scores in my purchasing decisions.

      • You’re quite right to say the boldest works are divisive. There are certainly great works with wide appeal, but those that are the most interesting, that examine the psyche and all that stuff, are probably also going to divide audiences. I also find those works more interesting to write about (and there’s more to say about them too, which helps.)

        As scores go, I definitely don’t put any stock in Metacritic or any single one of the big game sites’ ratings. That joke about nonsense IGN scores has been around for a long time now and for good reason. I don’t think any of us feel that a number can capture all the qualities of a game, good or bad — if a review is going to be of any use to me, it has to go into depth about why the game or whatever sort of work it is is good or bad or a mixed bag. Otherwise, what’s the point?

  6. Pingback: Real Neat Blogger Award – Frostilyte's Blog
  7. Pingback: August 2019 in Summary: Back on Track | Extra Life

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.