Tuesday, May 9, 2017

Story Lengths and Awards: When Does Size Matter?

In a May 7, 2017 blog post about the Hugo Award finalists for Best Novella, anthologist Rich Horton noted that "the print magazines were completely shut out of the Hugos this year" and wondered why that was the case even for novellas. Unlike in the novelette and short-story categories, it's not a case of free online stories competing with stories in magazines that you have to pay for. Nearly all the serious novellas published in 2016 had to be paid for.

I argued that it's because the print magazines have a lower cutoff for novella length, and that awards tend to go to longer stories--at least for short fiction.

But is that really true, and is it true in all three short-fiction categories?

Novellas

Let's look at the 2016 data for novellas (click charts to enlarge them):

Novellas in 2016
This chart shows the range of lengths for novellas originally published in 2016 and which were:
  • Published in one of the big three print magazines (Analog Science Fiction and Fact, Asimov's Science Fiction Magazine, and The Magazine of Fantasy & Science Fiction).
  • Published by Tor.com
  • Recommended by Rocket Stack Rank (that is, those with 4 or 5 stars)
  • Finalists for the Hugo Award.
  • Finalists for the Nebula Award
  • Published in any of the eleven magazines, eleven anthologies, and other sources that Rocket Stack Rank reads. It excludes "Penric's Mission" because it's really a short novel.
The top of each bar is the maximum length novella in that grouping. This shows that the print magazines never published anything over about 24,000 words in 2016. The number above that is the number of novellas in that category, so the print magazines published 22 novellas in 2016 and Tor.com published 23.

The bottom of each bar in the minimum length. This shows that none of the Hugo finalists was shorter than 27,500 words.

The white bar in the middle marks the average length. The overall average length for novellas was about 25,000 words.

Size Does Matter

From this data, both the Hugo and the Nebula awards strongly advantage length. (Even though there are only six finalists in each category, the difference between them and the overall category is statistically significant.)

F&SF''s submission guidelines say 25,000 max, while Analog and Asimov's both say 20,000.) Tor.com's guidelines, on the other hand, are 20,000 minimum and 40,000 maximum. This gave Tor a huge advantage.

But Why?

So are longer stories are just objectively better than shorter ones, on average? On the one hand, having more words gives the author has to explore his/her world, develop the characters, etc. On the other hand, that's also more space to bore the reader to tears.

Rocket Stack Rank's recommendations are not skewed by length in a significant way, so at least by our definition of quality, length really doesn't matter. That doesn't appear to match the opinion of the general public, as represented by the Hugo voters, though.

Playing Hard to Get Doesn't Work

An alternate possibility is that novellas in the print magazines are just too hard to get hold of. That is, if you heard that "The Ballad of Black Tom" was a great novella, you could easily go to Amazon.com and buy a copy. But if someone told you that "Wyatt Earp 2.0" was a great novella, you'd have to jump through hoops to get a copy of the January/February 2016 issue of Analog.

One the goals of Rocket Stack Rank was to help people jump through those hoops, but it's still probably the case that most people just don't bother.

If availability were the main factor, though, we'd expect the Hugo finalists to have the same distribution as Tor.com, but they don't, and this difference is also statistically-significant, albeit not as strong as between the Hugos and the print magazines.

Difficulty of acquisition likely does explain some of the lack of print-magazine nominees, but it doesn't seem to be the dominant effect. At least for this year, the people nominating for the Hugos and the Nebulas strongly favored longer stories.

Novelettes

That raises the question of whether we see anything comparable in the novelette or short-story categories. Looking at the same charts for novelettes, we see a different pattern:

Novelettes in 2016
This time only the Hugo Awards are different, biased up by about 2,500 words, and this difference is also statistically-significant. Here the difference cannot be accounted for by anything but a preference by Hugo voters for longer stories.

Probably the bigger surprise is how consistent all the other categories are.

Short Stories

Here's the same data for short stories:

Short Stories in 2016
In this category, there is very little difference in the word lengths among the different groupings. The only thing that really jumps out from this chart is that Tor.com seems to have a lower-limit of 2,000 words for short stories, whereas the print magazines will go down to just under 1,000. The shortest short story RSR read in 2016 was only ten words long, but almost nothing under 1,000 words got recommended or nominated for an award.
Note: we still don't have a word count for “An Unimaginable Light”, by John C. Wright, so the chart only shows data for five Hugo Finalists, not six.

Conclusion

This is strong (but not conclusive) evidence that Hugo voters strongly prefer longer stories to shorter ones at the novella length and at least somewhat prefer longer ones at the novelette length. We'd need to see this over several years to be confident that it's a general rule.

Future Work

Here are some thing we might look at in a future article:

  • Analyze several years worth of data to get stronger evidence.
  • Use the top-15 Hugo list, not just the 6 semifinalists.
  • Compare results from several different reviewers to see how much weight each one places on word length.
  • See if the Locus Recommended Reading List shows the same pattern.

And, of course, we welcome suggestions.

8 comments (may contain spoilers):

  1. For what it's worth, I did my best to estimate (inaccurately, as it depended on counting words in my Kindle) the length of "An Unimaginable Light", and came up with about 5600 words.

    ReplyDelete
    Replies
    1. Thanks. That'd bring the Hugo short-story average a bit closer to the RSR number. But the short-story numbers are already pretty close to each other.

      Delete
  2. I made a comment at Rich Horton's blog re cost and ease of acquiring magazines, but I see you've covered the difficulty aspect here already. The other one is cost - if I want to acquire a back issue of F&SF from Amazon UK to read e.g. The Vanishing Kind it's £6. The Dream Quest of Vellit Boe is £2.22. (Chosen because they're respectively my favourite magazine and tor.com novella from last year)
    Add to that the difficulty of even getting back issues for some of the mags that you note, which is going to get more eyeballs? Especially if you add in that people who are primarily novel readers will be seeing publicity and reviews for the tor.com novellas in their usual novel sources, but not so much the magazines. TOR have translated their novel sales tactics to novellas with great success.
    I'd agree that length is going to be an issue - it seems obvious that if two works come from equally talented writers then the one that gets 50% more space to develop it is likely to get a more successful result. It's not guaranteed, especially if the idea doesn't sustain the length, but tor.com seem quite relaxed about the word counts they accept so I imagine they're generally getting stories written to their natural length.

    ReplyDelete
    Replies
    1. If you only compare Hugo nominees to the Tor.com figures and ignore the print magazines, there's still a huge difference in favor of longer works. That's the strongest argument that availability alone doesn't explain things.

      But I agree that availability is a big deal. I just think it's secondary to length. At least for the Hugos this year.

      Delete
  3. I've been told that you cannot sell anything under 20,000 words for more than 99¢, but you can get $2.99 for works of 30,000 words or longer. In other words, if you're an independent writer, there is a vast difference at 30K.

    It suggests there might be logic in new categories: Short Story (under 10K) Long Story (10K but under 30K) Novella (30K but under 100K) and Novel above that.

    Not that we're going to abandon 50+ years of history for these categories, of course.

    ReplyDelete
  4. I don't think length is the deciding factor. Length was just incidental to the Hugo outcome imo.

    The biggest problem as already mentioned above is accessibility. I can buy paper copies of Asimov's and Analog at my local magazine shop, but getting older copies is cost-prohibitive. The cost of postage is more than the cost of the item.

    Most other genre magazines don't care where they sell to, so I can get e-copies of most magazines via Kobo. If they do care, it is blocked by region.

    The most inaccessible SF magazine I have come across is "The Magazine of Fantasy and Science Fiction", and I note very very few of their stories made it onto the Hugo short or long-list within these last 3 years.

    Five of my Short fiction nominations were TOR. Between being freely available online, paper copies available at the local library and available for purchase as an ebook in my region, they were easy to access. Plus each story often has eye-catching artwork that makes you stop and look just that little bit longer. It all helps.

    ReplyDelete
  5. Thanks for this. An excellent report. For some reason I'm obsessed with story lengths. I've also wondered what the shortest story to win or be nominated for a Hugo or Nebula. Candidates might be "Iyou were a dinosaur my love" by Brooke Bolander and Mantis wives by Kij Johnson.

    ReplyDelete
  6. “If You Were a Dinosaur My Love” was by Rachel Swirsky. Although Brooke Bolander also had a short Hugo and Nebula nominee with "Our Talons Can Crush Galaxies".

    ReplyDelete