The question of the month is “How Do We Rate Games?” LadiesGamers team members James and Elena discuss how they award the final verdict.
These Q&A articles are exclusively for Patrons. You can read the first two (with Yvonne and Paula) here and here.
“Liking” vs Scoring
Elena: I like our LadiesGamers rating system. It acknowledges that, as with other kinds of art—like movies—how we feel about a game is still pretty subjective. Our ratings, such as “I Like It” and “I Like It A Lot” reflect that subjectivity.
I would find it harder to rate games on a numbered scale, such as 5 or 10 point scale. I might spend too much time worrying over the precision of my rating, comparing one game against another.
James: I love the LG review system, as I’m pretty terrible with numbers. Sometimes I hover between scores to initially gauge if a game is good. In practice, this is often silly as no one game suits everyone and usually only the content of a review will tell you if the game will speak to you as a player or not.
Elena: Also, I think a numbered score might invite unnecessarily hasty judgment from some readers to dismiss a game if it didn’t have a “worthy enough” score. I confess that I myself often look at scores on Metacritic and Steam and decide too quickly, “This is game isn’t that great, huh? Maybe I’ll pass on it,” when I might actually have enjoyed it a lot.
James: There are many examples of games people love and adore that I would probably not score high. Because I’m weird.
I have reviewed games for a few different websites and LG is my favourite system. I never liked out of 5 or 10 star ratings. The LG community is also much nicer than I’m used to. When you add scores to games, some folk can be very rude when your opinion doesn’t match theirs. Since writing for LG, I have never had that happen, which is good.
Elena: We do have a very nice community. One of our colleagues noted that since we cover many indie games and don’t cover certain mainstream ones on PC, we’ve not had to deal with rabid fans.
Our Rating Criteria
Elena: So how do you decide on the final verdict, James?
James: To obtain an “I Like It,” games usually need to fulfill these criteria: release with minimal bugs, good controls, run well in TV and handheld modes, and feels like it would appeal to a wide audience even if the game doesn’t appeal to me specifically.
To obtain an “I Like It A Lot,” the game needs to do all the above but also offer a unique art style, good soundtrack, and a gameplay feel that is fun. Maybe even be a game that’s worth revisiting. Having an original concept that no other game has tried before also helps gain this rating from me.
“Two Thumbs Up” is generally awarded to games that speak to me specifically. The game usually does something particularly special that I end up thinking about it for days after playing. In a way, this is a rating one can’t aim for. Most games I give this rating to I stumbled upon, not expecting to like them as much as I did.
“I’m Not Sure” is usually given when the game, on release, is broken in places, requiring patches to fix. Or maybe it has gameplay flaws that make the experience not feel as good as it could be. I don’t consider this a negative rating; “I’m Not Sure” is more “on the fence.”
“I Don’t Like It”: I usually give this to games that are fundamentally broken and that I feel won’t appeal to most audiences even if the game is patched or fixed. Personally, I rarely like to give this rating but in some instances it is unavoidable.
Elena: For me, a game that’s fundamentally decent gets “I Like It,” even with obvious flaws that need improving.
“I Like It A Lot” goes to games with fun gameplay, no major problems, plus good art and/or music. I notice these games also tend to suit my taste in genre.
“Two Thumbs Up” is for games that score well in every department: gameplay, writing, art, sound (music doesn’t have to be super memorable, just not annoying), user interface and controls, plus quality-of-life features.
“I’m Not Sure” is for games that are somewhat OK and need much improvement. I wouldn’t recommend these games to other people just yet.
“I Don’t Like It” games? So far I haven’t encountered one… well, actually, I have. But I gave it leniency and a “Not Sure” rating, because I’m not sure it was designed to be a typical video game—the maker was an app developer using motion sensor technology to develop health and e-learning apps.
Going back to subjectivity: within our own group of LG reviewers, we probably have different internal criteria for a “Two thumbs up” etc. So ultimately it’s the full written review that tells you what to expect from a game, in terms of gameplay, difficulty, and so on.
James: Much of it comes down to the personal preference of an individual writer. Give a game to 10 writers and 7 writers will probably come out with similar ratings but the other 3 might be wildly different.
I like to think that readers like the individual personalities on our site. It’s what makes each of our review styles unique.
Reviews and Developers
Elena: I’ve sometimes had trouble deciding between “I’m Not Sure” and “I Like It”. I try to find reasons to Like It. Because I want to encourage developers to keep going. I assume that ratings of a new game could make or break sales for a small indie developer.
If a game element is bad, I try to give examples or reasons in my review instead of a mere blanket statement like “the music is awful.” So that if the developer is actually reading my review, it gives a concrete idea how to improve instead of unhelpful criticism.
James: I agree about trying to help out small indie devs by avoiding the “I Don’t Like It.” I never review a game with the intention of disliking it; I like to find the positives.
In general I have much more sympathy for small developers than big ones. But then again I don’t really review for many big developers. As you pointed out, I think giving reasons and explaining issues helps justify a rating.
Elena: This is a sidetrack, but what do you think of development companies that reward employees based on how good a review score they achieve, such as on Metacritic?
James: I’m not really in favour of rewards based on review scores, as how well a game scores does not correlate to how hard employees work. Many games that review poorly have dedicated, hard-working staff. But it’s a complicated discussion, as targets can be healthy for some people and aiming to make a game that reviews well is good. I just don’t believe that staff should stress over these things if it causes harm to their mental health.
I often feel there is a big disconnect between reviewers and how games are made. Many reviews seem to focus on the product, which is fine, but reviewers often forget that games are not easy to make. It’s hard to make such a complicated project come together well. This leads back to me disliking the numbered score system.
Overall, in short, no: I don’t think review scores should tie to bonuses. But it’s a complicated issue. At the end of the day, a bonus is hard to obtain if a game doesn’t sell and generally good reviews help with that.
Elena: You mentioned a big disconnect between reviewers and how games are made. Did you mean that reviewers can be too harsh, or something else?
James: Yes, I meant some reviews (not all) are a bit harsh on games. Then again, it’s difficult to say if a review should factor in how games are made. I think readers simply want to know if a game is good, rather than reading about the whole production process.
Elena: It does justice to a game to talk about how the design works well (or not). Or how the design differs from other games in its genre. But while I like to hear about the development, I think it’s best if a review generally sticks to the subject of whether a game is good. There’s a distinction between how games are made and how games work. Reviews address how they work.
I’m glad that on LG we’re starting to feature more interviews with indie developers, to hear about how games are made! So we get to scratch that itch too.
James: I like developer interviews, but it’s sometimes very hard to get them. Folk are always so busy.
Final Thoughts on Rating Games
Elena: Any final thoughts before we wrap up for today?
James: When I review games, I try and make the review my own take on the title. To achieve this, I don’t read other reviews of the game, hopefully to put my own unique spin on it.
Elena: Yeah, if possible, I try to avoid reading other reviews too, until mine is done.
James: One final thought. Advice to anyone reviewing: Find your own writing style, be yourself but also be kind to your peers, and don’t hesitate to ask for help if you get stuck.
Elena: Nice. My final thoughts: Recently I learned to not rush a review—if I don’t have a deadline, that is!
While playing 80 Days, I found myself enjoying the game more when I took time to savor it and play it on and off, instead of bingeing on the game to get a review out sooner. If I had tried to publish a review earlier, I would have given it a lower rating.
So here’s something nice about reviewing for LadiesGamers. While for some games we have a recommended deadline for publishing (roughly 2 weeks after receiving a review copy), other times we can review games at a leisurely pace too, for instance when we bought the game ourselves!