Getting More From Short-Fiction Reviews

What's Wrong With Short-Fiction Reviews?

About two weeks after we started working on Rocket Stack Rank, Neil Clarke, editor of Clarkesworld Magazine, wrote an editorial titled The Sad Truth About Short Fiction Reviews, in which he said that his magazine doesn't offer short-fiction reviews because "they don't have much value." That was a little discouraging, but he also said something encouraging:

Could it be that most people are looking for “read this,” rather than “here’s a bunch of stories I read?”

After reading over 200 short stories and reading up to six online reviews for each one, we think most online reviews are really written to be read after you have read the story. Their target customer seems to be the magazine subscriber who routinely reads the whole magazine and then turns to a reviewer to get other opinions on what he/she just read.

As a result, they give too much away in their reviews; they risk spoiling interesting bits of the story if you haven't read it yet. Secondly, some reviewers make it hard to tell whether they recommend a given story or not. In fact, some reviewers simply like all stories; they only say good things about all (or almost all) of the stories they review. Third, their recommendations aren't collected into a single place; there's no list of "this is what we're recommending for 2015," so a reader has to page through dozens of blog posts per reviewer.

If you are a casual SF/F fan looking for suggestions for stories to read, Neil Clarke is right: it's hard to get much use out of existing short-fiction reviews. People need a recommendation system, not just a set of reviews.

What Should a Short-Fiction Recommendation System Do?

To help the person trying to find good stories to read, we think a recommendation system should do the following:

1. Point the person to a few stories recommended by one or more reviewers, offering a general idea of the topic of each story without giving anything away that spoils the story.
2. Tell them how to get hold of a copy of the story.
3. Offer a review of the story meant to be read after the reader has finished it. Not a page-long literary critique, but something that invites further discussion.
4. Provide a place for discussion of the story.

This is independent of what sort of criteria determine the story ratings.

In this view, many SF/F reviewers are providing step 3 and nothing more. We designed RSR to do all four, but step 3 is our weakest link, since we're professional software guys (and long-time fans) but we're not professional reviewers. (A future post will discuss how our ratings compare with the professionals.)

How Do We Get Reviews into a Recommendation System?

We've identified the online reviewers who do actually make recommendations. For each story that we review (no matter what rating we gave it) we check to see who (if anyone) recommended the story, and we provide links to their recommendations. We put them in red to make them easier to find. (Look at the Year-To-Date list to see what we're talking about.)

This means that you can glance down the list of stories and choose based on your favorite reviewer(s). Or you can mix and match. For example, you might decide that you want to read all the stories that Lois Tilton recommended, but only if RSR gave them at least three stars. With a little experimentation, you should be able to figure out which reviewers best match your own tastes.

Conclusion

Mr. Clarke has a point, but nevertheless there is a lot of value in the short-fiction reviews that are already on the web. A good recommendation system should let readers unlock that value.