Story Lengths and Awards: When Does Size Matter?

In a May 7, 2017 blog post about the Hugo Award finalists for Best Novella, anthologist Rich Horton noted that "the print magazines were completely shut out of the Hugos this year" and wondered why that was the case even for novellas. Unlike in the novelette and short-story categories, it's not a case of free online stories competing with stories in magazines that you have to pay for. Nearly all the serious novellas published in 2016 had to be paid for.

I argued that it's because the print magazines have a lower cutoff for novella length, and that awards tend to go to longer stories--at least for short fiction.

But is that really true, and is it true in all three short-fiction categories?

Novellas

Let's look at the 2016 data for novellas (click charts to enlarge them):

Novellas in 2016

This chart shows the range of lengths for novellas originally published in 2016 and which were:

• Published in one of the big three print magazines (Analog Science Fiction and Fact, Asimov's Science Fiction Magazine, and The Magazine of Fantasy & Science Fiction).
• Published by Tor.com
• Recommended by Rocket Stack Rank (that is, those with 4 or 5 stars)
• Finalists for the Hugo Award.
• Finalists for the Nebula Award
• Published in any of the eleven magazines, eleven anthologies, and other sources that Rocket Stack Rank reads. It excludes "Penric's Mission" because it's really a short novel.

The top of each bar is the maximum length novella in that grouping. This shows that the print magazines never published anything over about 24,000 words in 2016. The number above that is the number of novellas in that category, so the print magazines published 22 novellas in 2016 and Tor.com published 23.

The bottom of each bar in the minimum length. This shows that none of the Hugo finalists was shorter than 27,500 words.

The white bar in the middle marks the average length. The overall average length for novellas was about 25,000 words.

Size Does Matter

From this data, both the Hugo and the Nebula awards strongly advantage length. (Even though there are only six finalists in each category, the difference between them and the overall category is statistically significant.)

F&SF''s submission guidelines say 25,000 max, while Analog and Asimov's both say 20,000.) Tor.com's guidelines, on the other hand, are 20,000 minimum and 40,000 maximum. This gave Tor a huge advantage.

But Why?

So are longer stories are just objectively better than shorter ones, on average? On the one hand, having more words gives the author has to explore his/her world, develop the characters, etc. On the other hand, that's also more space to bore the reader to tears.

Rocket Stack Rank's recommendations are not skewed by length in a significant way, so at least by our definition of quality, length really doesn't matter. That doesn't appear to match the opinion of the general public, as represented by the Hugo voters, though.

Playing Hard to Get Doesn't Work

An alternate possibility is that novellas in the print magazines are just too hard to get hold of. That is, if you heard that "The Ballad of Black Tom" was a great novella, you could easily go to Amazon.com and buy a copy. But if someone told you that "Wyatt Earp 2.0" was a great novella, you'd have to jump through hoops to get a copy of the January/February 2016 issue of Analog.

One the goals of Rocket Stack Rank was to help people jump through those hoops, but it's still probably the case that most people just don't bother.

If availability were the main factor, though, we'd expect the Hugo finalists to have the same distribution as Tor.com, but they don't, and this difference is also statistically-significant, albeit not as strong as between the Hugos and the print magazines.

Difficulty of acquisition likely does explain some of the lack of print-magazine nominees, but it doesn't seem to be the dominant effect. At least for this year, the people nominating for the Hugos and the Nebulas strongly favored longer stories.

Novelettes

That raises the question of whether we see anything comparable in the novelette or short-story categories. Looking at the same charts for novelettes, we see a different pattern:

Novelettes in 2016

This time only the Hugo Awards are different, biased up by about 2,500 words, and this difference is also statistically-significant. Here the difference cannot be accounted for by anything but a preference by Hugo voters for longer stories.

Probably the bigger surprise is how consistent all the other categories are.

Short Stories

Here's the same data for short stories:

Short Stories in 2016

In this category, there is very little difference in the word lengths among the different groupings. The only thing that really jumps out from this chart is that Tor.com seems to have a lower-limit of 2,000 words for short stories, whereas the print magazines will go down to just under 1,000. The shortest short story RSR read in 2016 was only ten words long, but almost nothing under 1,000 words got recommended or nominated for an award.
Note: we still don't have a word count for “An Unimaginable Light”, by John C. Wright, so the chart only shows data for five Hugo Finalists, not six.

Conclusion

This is strong (but not conclusive) evidence that Hugo voters strongly prefer longer stories to shorter ones at the novella length and at least somewhat prefer longer ones at the novelette length. We'd need to see this over several years to be confident that it's a general rule.

Future Work

Here are some thing we might look at in a future article:

• Analyze several years worth of data to get stronger evidence.
• Use the top-15 Hugo list, not just the 6 semifinalists.
• Compare results from several different reviewers to see how much weight each one places on word length.
• See if the Locus Recommended Reading List shows the same pattern.

And, of course, we welcome suggestions.