To improve our guidelines for rating stories, we're studying all the stories we rated in 2015 and comparing our ratings to the ratings from other reviewers. The biggest problem with our system seems to be that our three-star stories include a number of memorable tales that didn't earn four stars solely because they had simple plots. Accordingly, we're changing our guidelines for 2016 to give four stars to any well-written story that either has a sophisticated plot or is highly memorable. Five-star stories still require both.
Although "memorable" seems rather subjective, a look back through the three-star stories from 2015 suggests that it's fairly easy to identify memorable stories (you always know what part stuck with you) even if it's hard to describe a general rule for what makes something memorable. Also, provided the story at least met the 3-star bar, a memorable story usually got at least one recommendation from another critic and generally ended up in one or more year's-best anthologies. So it actually appears that "memorable" really is a workable and useful criterion for rating stories.
(The blue portion of each bar is how many stories in that category got a recommendation from at least one other reviewer besides RSR.)
It's not essential for the graph to be balanced, of course, but it's a hint as to whether we're judging stories too harshly or too leniently. We definitely had plenty of room to move stories from 3 to 4 stars.
There's a certain elegance to saying that a 4-star story is sophisticated or memorable while a 5-star story is sophisticated and memorable. Also, it's hard to write a story with a sophisticated plot (and there aren't many) but it's also hard to write a memorable story. Both are worth calling attention to, even if a Hugo-worthy story should have both.
We upped the ratings of the following three stories already reviewed in 2016:
Razorback, by Ursula Vernon
The Savannah Liars Tour, by Will McIntosh
Theories of Mind, by Conor Powers-Smith
Memorable Three-Star Stories
It was painful to give just three stars to a story that, subjectively speaking, we really did want to recommend that people read. It felt wrong not to recommend it just because we probably wouldn't consider it for a Hugo Award.Although "memorable" seems rather subjective, a look back through the three-star stories from 2015 suggests that it's fairly easy to identify memorable stories (you always know what part stuck with you) even if it's hard to describe a general rule for what makes something memorable. Also, provided the story at least met the 3-star bar, a memorable story usually got at least one recommendation from another critic and generally ended up in one or more year's-best anthologies. So it actually appears that "memorable" really is a workable and useful criterion for rating stories.
Missing Four-Star Stories
Separately, our distribution of ratings was rather lopsided; we were shooting for a symmetrical distribution, but in fact there were over twice as many two-star stories as four-star stories.
(The blue portion of each bar is how many stories in that category got a recommendation from at least one other reviewer besides RSR.)
It's not essential for the graph to be balanced, of course, but it's a hint as to whether we're judging stories too harshly or too leniently. We definitely had plenty of room to move stories from 3 to 4 stars.
The Proposed Change
If we changed the guideline to say that a 4-star story must have a sophisticated plot or be memorable, we would move about 50 of the 2015 stories from 3 stars to 4 stars, and almost all of those would be stories that were recommended by other reviewers.There's a certain elegance to saying that a 4-star story is sophisticated or memorable while a 5-star story is sophisticated and memorable. Also, it's hard to write a story with a sophisticated plot (and there aren't many) but it's also hard to write a memorable story. Both are worth calling attention to, even if a Hugo-worthy story should have both.
Conclusion
The change makes sense, is easy to apply, and fixes two problems at one stroke. It's too late to revise the ratings for 2015, but we'll apply this scheme going forward.We upped the ratings of the following three stories already reviewed in 2016:
Razorback, by Ursula Vernon
The Savannah Liars Tour, by Will McIntosh
Theories of Mind, by Conor Powers-Smith
No comments (may contain spoilers):
Post a Comment (comment policy)