Saturday, September 26, 2015

Finding Hugo-Worthy Short Fiction

(Update: 2/12/16 RSR now reads 11 magazines, up from 6 magazines in 2015. We also read select works from other sources.)

For many years, if you wanted to read the short fiction most likely to be nominated for the Hugo and Nebula awards, you subscribed to the top three print magazines: Analog Science Fiction and Fact, Asimov's Science Fiction Magazine, and the Magazine of Fantasy & Science Fiction. Eighty percent of the nominated stories typically came from one of those three.

Today the situation is more complicated, so we looked at the last fifteen years worth of data to get an idea of just how complicated it really is. The big three print magazines alone are definitely not enough anymore, but we found we could get at least 50% coverage if we added just three online magazines: Clarkesworld Magazine, Lightspeed Magazine, and The results are detailed in the charts below.

Hugo Finalists

Here is where the Hugo Finalists (the five nominees on the ballot for each category) came from over the past fifteen years:

This chart combines short stories, novelettes, and novellas. The non-magazine category comprises anthologies, collections, and "singles" (stories sold individually, usually novellas). Here are a few observations:
  • The big three (AnalogAsimov's, and F&SF) really dominated the short-fiction Hugos up until about six years ago.
  • Tor and Clarkesworld have made a big difference during the last six years. Lightspeed, not so much.
  • The last three years were very different from all the rest, and it cannot all be ascribed to slates.
  • Analog has almost fallen off the charts. All six Analog finalists in the past three years came from slates.

Nebula Finalists

The Nebulas aren't judged the same way as the Hugos, of course, but they're slate-proof, so they can give us a rough idea whether something really dramatic happened to the magazines over the last three years.

This chart is constructed the same way as the Hugos chart. Here are a few observations:
  • Lightspeed is a much bigger factor for the Nebulas than for the Hugos.
  • Non-magazine sources are a smaller factor.
  • In the past six years, the big three lost a lot of share to Tor, Clarkesworld, and Lightspeed.
  • In the past two years, the big three were crushed.


Someone limiting him/herself to just these six sources is very likely to see at least half of the stories that end up as either Hugo or Nebula finalists. To get that back up to 80%, one would need to include some original anthologies, collections, and/or singles; adding more magazines isn't likely to make as much difference.

Nevertheless, the past two years do seem to have been different from earlier years, and not just because of slate voting. These three online magazines are definitely giving the print ones very serious competition. This may not be entirely due to quality: as far as getting nominations goes, online magazines have an edge over print ones because fans can easily exchange links to online stories.

The eclipse of Analog merits attention, particularly given its popularity with slates. Many of the people promoting slates have argued that the big awards have quit rewarding their type of story. While these charts don't support claims of some conspiracy, they do suggest that something really has changed over the past decade or so.

Future posts will look at the influence of Locus Recommended Reading Lists and of popular anthologies.

Rocket Stack Rank includes information on how to subscribe to the key Science Fiction and Fantasy magazines, including how to obtain back issues.

Update 2/14/2016

In 2015 we actually read and reviewed every story in all of the six magazines above plus nine highly-reviewed anthologies and a dozen or so stand-alone novellas. We also reviewed selected stories from other sources. We won't know how this compared to the Hugo and Nebula finalists for a few months, but it covered over 2/3 of the Locus Recommended Reading List in the short-fiction categories. Not bad, given that we didn't start until the end of August 2015.

For 2016, we have added Apex, Beneath Ceaseless Skies, Interzone, Strange Horizons, and Uncanny to our list of magazines, and we expect to read at least twelve anthologies in 2016. We continue to be open to people directing us to outstanding stories from other venues, although we can't promise to read everything.

Because some people were confused, we changed the wording in the first paragraph of the conclusions from "adding more magazines isn't likely to make much difference" to "adding more magazines isn't likely to make as much difference." That is, before adding more magazines, we needed to cover original anthologies and selected stand-alone novellas.

7 comments (may contain spoilers):

  1. Cool analysis! I found some similar things in a related analysis here:

    Besides the ones you highlight, Subterranean, Strange Horizons, and Interzone have drawn a pretty good number of nominations, Locus recommendations, and "best of" selections.

    1. Nice web page! Yes, we noticed that Subterranean had done very well, but, sadly, they're gone now. We might consider adding one or two magazines next year, but I think the next frontier for us will be finding a way to include the most promising anthologies.

  2. Finding a useful angle into the anthologies seems like a worthwhile endeavor!

    1. Best idea we've got so far is to look at which anthologies the various finalists have come from over the past several years and then to see who the editors were. It may turn out that two or three anthologists account for most of it.

      If you have a better idea (or even some data) we'd love to hear it!

  3. Greg,

    Just to say, I thought you made an interesting argument on my blog about why you would expect the best stories to be in the top six magazines.

    I was thinking about anthologies and individual collections. I'd expect the quality of stories to be higher than an average story in the 'Top Six' because many anthologies are invitation only (or so I gather) and individual collections also favour established authors. They're also easier to produce and distribute with digital/online publishing, but likely to be underrepresented compared to the magazines because they're one-offs and not free.

    Does the percentage of non-magazine finalists increase over time? I can't tell from your graph.

    [NB: I deleted the blogpost because File770 had generated tens of comments speculating (mostly inaccurately) about my motivations, etc. Thank you for being so welcoming to the fan community and encouraging me to read, ask questions and get involved].


    1. The percentage of non-magazine Hugo finalists goes up and down a lot over time. I don't think it's possible to claim there's any real trend. The number on the Locus Recommended Reading List does seem to be trending up.

      Now that we're caught up on short fiction for the year from our six magazines, we could look at a few anthologies. We're just trying to decide which ones are the most promising.

  4. I should clarify - the comments were on File770 and not the blog.