Sunday, April 23, 2017

Fix the Slating Problem Forever

This year's Business Meeting has an opportunity to fix the "slating problem" forever by adopting Three-Stage Voting (3SV) and either retaining E Pluribus Hugo (EPH) or adopting E Pluribus Hugo Plus (EPH+).

We did extensive analysis of nominating data from 2014 to 2017 to help fans planning to attend the Business Meeting at WorldCon75 answer the following questions:
  • Should we endorse continued use of EPH?
  • Should we ratify EPH Plus?
  • Should we ratify 3SV?
  • Is any of this really going to fix the slating problem in the long term?
For each year, we estimated how much impact different sized slates would have had under six different combinations of voting systems, assuming optimal strategies by the slate organizers. We found that EPH+ and 3SV are individually too weak to fully protect the Hugo Awards from future slating campaigns, but in combination they're unbeatable.

1. What Does All of This Even Mean?

Those who understood everything in the section above should skip to the analysis below. For those who came in late, here's a quick summary.

A Bit of History

Take a quick look at the "Background" section in last-year's article, Slate Voting Analysis Using EPH Data: 2014-2016. That'll explain what a "slate" is and what the EPH (E Pluribus Hugo) voting system is. The short summary is that, starting in 2014, a small group of disaffected people starting exploiting a flaw in the nomination system for the Hugo Awards. By agreeing to nominate in lockstep from a small slate of approved works (whether they had read them or not), a group of just 200 people in 2015 and 400 in 2016 were able to grab most of the finalist slots on the final ballot.

This chart shows the percentage of people nominating who supported the slates.


A small but not insignificant minority. The next chart shows what fraction of the finalists they captured. (These numbers ignore withdrawals and disqualifications, and they include "hostages," which are nominees that would have made it to the ballot even without support of the slates.)


In other words, a small minority was able to capture almost all the slots on the final ballot up until rule changes made that more difficult. The 2017 numbers are deceiving, though; the slaters could have done much better had they really been trying, as we'll see later on.

This group is usually called "the puppies" but we'll call them "slaters" because anyone could choose to use the same tactics at any time and for any reason.

The reason the slated nominees didn't win on the final ballot in 2015 and 2016 was that the rules allow members to vote for "No Award" as well as any finalist. If No Award wins on the final ballot, then no award is given that year. In the 50+ years before 2015, No Award had won only 5 times, but the 2015 Convention used it 5 times all by itself--including for key categories like Best Short Story. The only reason 2016 was better was that the slaters made some strategic errors that year.

For details on this history, read Sci-Fi’s Hugo Awards and the Battle for Pop Culture’s Soul, by Amy Wallace (Wired Magazine; October 30, 2015).

Here are a few other terms that'll be useful to know:

The World Science Fiction Society (WSFS). That sounds fancy, but it's really just all the fans who bought memberships to the current convention. Members can be attending members, who actually go to the convention, or supporting members, who get to vote, but don't actually go to the convention. It's a whole lot cheaper to be a supporting member. Both kinds of member get free copies of most of the finalists, which makes the cost of a supporting membership a bargain in many people's eyes.

WorldCon75. That's the upcoming convention, which opens August 9, 2017 in Helsinki, Finland.

The Business Meeting. This is actually a sequence of daily meetings held at each annual convention (e.g. WorldCon75) which have the power to change the WSFS constitution. Any attending member can show up, make motions, and vote. Supporting members have no voice in the Business Meeting.

The WSFS Constitution is a long and complicated document, but for our purposes it's important because it contains the rules for nominations and final voting on the Hugo Awards. (Article 3, starting at section 3.7.)

Fixing the slating problem means changing the voting system, and that means amending the constitution. To do that takes two years; one Business Meeting proposes an amendment and then the next one has to ratify it. This is why fixing the "puppy problem" has taken so long.

Five of Six (5/6) is a simple voting system that says that every member can nominate 5 items. WSFS counts up how many votes each nominee got, and the top 6 become finalists. The old rules (which were in effect before 2017) allowed 5 nominations but only produced 5 finalists.

E Pluribus Hugo (EPH) is a complex voting system proposed by actual experts in election systems. It attempts to deal with slates automatically. In loose terms, it does this by diluting the influence of everyone's votes based on how many nominees they have in the final list. A normal candidate is diluted by a factor of 1.2 to 1.6, but a full slate of 5 would be diluted by a factor of 5.

More precisely, if you have 5 nominees, then each one gets one vote but only 1/5 of a point. If you just had 4 nominees, each one still gets one vote but 1/4 of a point. Per category, the whole list of everyone's nominees is sorted by points, and the bottom two "fight it out." Whichever one has the most votes (not points) survives, and the other is discarded. As a consolation prize for everyone who had nominated the loser, the point totals for their surviving nominees (if any) are recomputed. So when one of your 5 nominees is eliminated, your other 4 automatically go from 1/5 point each to 1/4 point each. The list is resorted, and the process repeats until only 6 candidates remain.

This has the effect that it's very hard for a slate to capture all of the finalists. It's elegant in that it doesn't depend on anyone deciding what is and isn't a slate.

The 2015 Business Meeting at Sasquan in Spokane, WA proposed amending the constitution to use Five of Six and/or EPH.  The 2016 Business Meeting at MidAmeriCon II in Kansas City, MO, ratified both of them. They were used for the 2017 nominations.

But this process was very contentious. When EPH was ratified in 2015, most people believed it would limit the slates to just one nominee out of 5. Between the two business meetings, it was learned that for most categories, it would probably limit the slates to 3 or 4 out of 5--despite them only accounting for 10-15% of the number of people making nominations. Many people felt this was too much complexity for too little return.

To address this problem, two more amendments were proposed.

EPH Plus (EPH+) modifies the way EPH dilutes votes. The logic for doing this is complex, but the change is simple. Instead of diluting by factors of 1, 2, 3, 4, 5, 6 it dilutes by factors of 1, 3, 5, 7, 9, 11. So under EPH, if you have three nominees in the list, you give each one just 1/3 of a point, but under EPH+ each one only gets 1/5 of a point.

Calculations on the data from 2014 suggested that this would limit slate to 3 or 4 finalists, not 4 or 5. Together with 5/6, fans would at least get 3 legitimate (aka "organic") nominees per category.

Three-Stage Voting (3SV). Most people felt that giving up half the nominees to slate candidates wasn't acceptable. Accordingly, they proposed a third stage of voting in between the nomination stage and the final vote. The idea was to allow people to see the top-15 nominees and let members vote No Award before producing the list of finalists. There are some complicated thresholds meant to make this hard to abuse.

The 2016 Business Meeting approved both EPH+ and 3SV. (Exact details are in the minutes.)

So what does the 2017 Business Meeting need to decide at WorldCon75?

First, it needs to decide whether EPH and/or 5/6 actually worked. If not, it has the power to suspend either one of them for a year. That is, it could keep either or both of them from being in effect in 2018 at the San Jose convention.

Second, it needs to decide whether to approve EPH+ and/or 3SV.

Now go back to the top and see if all those items make sense now.

2. What should the Business Meeting at WorldCon75 do?

Summary

We analyzed four years (2014-2017) of nomination data to estimate the impact that optimally-organized slates would have had if any of six different voting systems had been in effect those years. All six systems included 5/6. That is, they assumed members get to nominate 5 candidates in each category and the final ballot will comprise 6 finalists. To 5/6 we added the following:
  1. Std (nothing but 5/6)
  2. EPH
  3. EPH+
  4. 3SV
  5. 3SV/EPH
  6. 3SV/EPH+ 
For each year, we produced two theoretical maximum graphs. A "finalist graph," which shows what percentage of finalists a slate could have captured for a given number of slate voters, and a "sweeps" graph, which shows what percentage of entire categories a slate could have captured.

Looking at those four pairs of graphs (2.1-2.4 below), we will draw the following conclusions;
  • Std (5/6) by itself is far too weak.
  • EPH doesn't protect enough finalists, but it is excellent at preventing sweeps.
  • EPH+ is an improvement on EPH, but it's still not enough by itself.
  • 3SV is much stronger for protecting finalists, especially for modest numbers of slate voters, but it's vulnerable to sweeps, and it breaks down for slates above about 300 people.
  • The 3SV/EPH and 3SV/EPH+ combinations are far, far stronger than either component alone. Either combination is probably sufficient, but the second one is stronger.
Accordingly, we conclude that the Business Meeting should ratify both EPH+ and 3SV. That should protect the Hugos from slating interference for the nonce.

2.1 Theoretical Maximum Slate Impact on 2014

That year, the slates (Sad Puppies only) had about 100 slate voters and they got 10 out of  102 possible top-6 slots, or about 10% (or 12% of the 85 slots they were trying for, under the then-current rules). Here is the finalist graph for that year, which shows the most they could have achieved, had these various systems been in effect at the time:


From the finalist graph, we can conclude that:
  • For the 100 slaters we actually had that year:
    • Under the 5/6 rules, if they had played their cards right, they'd have taken over 60% of the finalist slots.
    • EPH would have limited that to 30%. 
    • 3SV would have eliminated almost all of it.
    • Either combo would have eliminated the slates entirely.
  • For 200 slaters (the number we had in 2015):
    • 5/6 would have given them over 90% of the finalists. 
    • EPH, EPH+, and 3SV would have limited them to between 40% and 60%.
    • The combination of 3SV with either EPH or EPH+ would have shut the slates out.
  • For 400 slaters (as we had in 2016):
    • 5/6 would have given almost all the finalists to slate candidates.
    • 3SV would have given up over 80%. (3SV is weaker than EPH for large numbers of slate voters.)
    • EPH would have given up over 75%.
    • EPH+ would have held the slaters to "only" 65%.
    • 3SV/EPH would have let 20% of them through.
    • 3SV/EPH+ would have stopped all but 5% of them.
This is the easiest case for the slaters because it had the fewest organic nominations of any of the four years we studied.

Now let's look at the sweeps graph.


Observations:
  • 100 slate voters would have swept 20% of all categories with 5/6
  • 200 slate voters would have swept 75% under 5/6 but only 5% under 3SV.
  • 400 slate voters would have swept 90% of categories under 5/6, 50% under 3SV, and 20% under EPH, but only 5% under EPH+.
  • 3SV/EPH and 3SV/EPH+ would have prevented any sweeps give 500 slate voters or fewer.
This is where the EPH logic really shines; it is a powerful force against anything sweeping an entire category. 

2.2 Theoretical Maximum Slate Impact on 2015

That year, there were 200 actual slate voters (Sad and Rabid combined) and they captured 67 of the top-6 slots for a total of 66%. Note that under the rules for 2015, they were only trying to get 5 per category, and that's almost 79% of the top-5 finalist spots. Here is the finalist graph showing what a better-organized slate could have done in 2015:


The graph tells the following tale:
  • For 100 (the number of slaters in 2014 and 2017):
    • 5/6 would have let the slates have about 55% of the finalist slots.
    • EPH would give up 30%.
    • EPH+ would give up 20%.
    • 3SV would only give up 5%.
    • The combos wouldn't give up anything.
  • For 200 slate voters (which we actually had that year)
    • Under 5/6, if they played their cards right, they could have had 90%, not just 66%.
    • EPH would have held them to 50%.
    • EPH+ and 3SV would have tied at 35%.
    • The two combos would have shut the slates out almost completely.
  • For 400 (as we had in 2016):
    • 5/6 would have given the slates everything.
    • EPH, EPH+, and 3SV would all give up more than half.
    • The combination of 3SV and EPH would give up just 13% of the finalists.
    • The combination of 3SV and EPH+ would give up only about 5%.
Now let's look at the 2015 sweeps graph:


Observations:
  • Having more voters made it a little harder for slates to sweep, but not much.
  • EPH and EPH+ benefited more from the extra nominating voters than 5/6 or 3SV did.
  • EPH+, 3SV/EPH and 3SV/EPH+ are invulnerable under 500 slate voters.
The 2015 results may be the best guide to the future, since, other than the slates, that was a normal year for nominations. In 2016, a vast number of people turned out to oppose them, making it a less-realistic comparison.

2.3 Theoretical Maximum Slate Impact on 2016

In 2016, we saw the biggest turnout of fans nominating for the Hugos ever, but we also saw 400 slate voters (essentially all Rabid Puppies). Despite the big turnout, the slates took 76 of the top-6 spots, for 75% of the total. Only counting the top-5, they took 72 out of 85 or 85%.

It's worth noting that the above paragraph somewhat overstates the success of the slates in that they deliberately nominated a number of "hostages," which were works that would have made the finalist lists without help from the slates. If you subtract 11 such works, their success rate falls to 63% (for top 6) and 72% (for top 5).

Here's the finalist chart:


Looking at the chart:
  • With this big a voter turnout, 100 slaters wouldn't get even 10% of the finalists under any system.
  • For 200 slaters:
    • 5/6 would have given them 30% under 5/6.
    • EPH would have given them 20% 
    • EPH+ would have given them 13%
    • 3SV would have given them 5%
    • Either combo would have given them nothing.
  • For the 400 slaters we actually had:
    • 5/6 would have given them over 80% of finalists.
    • EPH would have held them to 50%, 
    • EPH+ would have held them to 40%.
    • 3SV would have held them to 20%.
    • Both combo methods would limit them to 1% (one finalist in one category).
The sweeps chart shows a similar effect.


Even with this enormous turnout, 5/6 would still have given up 35% of all categories to slates, but 3SV would have done so only for 6% (two categories).

If we could be sure that every year would be as big as 2016, 3SV alone might be good enough--depending on how many slots one is willing to concede to the slates. Unfortunately, it would be hard to repeat 2016 without another disaster like 2015.

2.4 Estimated Theoretical Maximum Slate Impact on 2017

We won't have all the data for 2017 until August after the Hugo Awards have been presented, but we wanted to at least take a stab at analyzing this year's numbers now. You can read details below on how we filled in the holes. View this as an experiment more than anything else.

We estimate there were about 100 slate voters (all Rabid Puppies) in 2017. They captured only 16 out of 108 possible top-6 finalist slots, for just 15%. This time 5/6 was in effect, so they weren't targeting 5 finalists per category anymore. However, six of their nominees were things that other people might have nominated anyway, so 10% is a more realistic estimate of their success in 2017. Here's the finalist chart:


Based on the chart, we can say:
  • For 100 slaters:
    • 5/6 would have given them 20%
    • EPH and EPH+ would have limited them to 10%.
    • Everything else would have shut them out completely.
  • With 200 slaters:
    • 5/6 would have let them take 70% of the finalists.
    • EPH would give up 40%
    • EPH+ would give up 20%
    • 3SV would only give up 10%.
    • The combos would both shut them out.
  • With 400 slaters:
    • 5/6 would give up about 95% of all finalist slots.
    • EPH would give up over 60%.
    • 3SV would give up 55%.
    • EPH+ would give up 45%.
    • The combos would give up almost nothing.
Compared to the charts for the previous years, this seems to underestimate the impact of a well-organized slate by as much as a factor of 2. In particular, the slates actually did get 10% this year with a desultory effort. it makes sense that a serious effort would net them 20%, as the 2015 numbers imply.

Even so, it makes a strong argument for approving one of the combination methods--not replying on EPH(+) or 3SV alone.

The sweeps chart is therefore likely to be a bit too optimistic.


Even so, we think few would consider allowing a slate of 400 people to sweep 10% of all categories acceptable.

In August, when the final numbers come out, we'll update this with the actuals and then we'll see how good an estimate this really was.

Conclusion

We cannot go back to a simple system. Even very small numbers of slate voters easily overwhelm the old approval voting system, even with 5/6.

EPH alone (meaning EPH combined with 5/6) is not enough. In a normal year, 200 slaters (10% of the people nominating) could grab half the nominations. It excels in preventing sweeps, though. Even if we change nothing, we are probably safe from having to give No Award again.

EPH+ is better, but probably not enough better. (One third instead of one half.)

3SV is slightly better than EPH+ for smaller numbers of slaters, but it falls behind when there are more than 200 of them, and it is much more vulnerable to sweeps.

The only thing that gives complete protection is a combination of 3SV and either EPH or EPH+. Given our experiences the past three years, it seems wise to err on the side of caution and approve both 3SV and EPH+.


APPENDIX: Details of Analysis

For those who want to know exactly how we came up with these numbers, read on.

A.1 Computing the Finalists and Sweeps Charts

To compute the numbers for the charts, we needed the organic vote totals (that is, excluding the slate votes) for ranks 1-6 in each category for each year. We also needed the "EPH Deflator" for each rank 1-6 in each category. The EPH Deflator is just the ratio of the total votes to the total points--it's a measure of how much EPH has "deflated" the votes. The challenge is that not all of this infomation is directly available.

A.1.1 Vote Totals

To eliminate the effects of the slates, we used the 20142015, and 2016 Hugo Nominations Organic Estimates, which we computed last year. For the 2017 vote totals, we used the published numbers for the #1 and #6 finalist in each category and then interpolated the rest, as described in our article Slating Analysis: 2017.

A.1.2 EPH Deflator

To get EPH deflator for each category, we again used the numbers from the 20142015, and 2016 Hugo Nominations Organic Estimates. For 2017, where that data won't be available until August, we used the median deflator for each category across the previous three years and applied it to all ranks.

For the new Best Series category, we used the deflator from Best Novel.

A.2 Slate Impact on Finalists

A.2.1 Some Terminology

A few definitions will make this discussion a lot easier to follow. 

voter is anyone who nominated anything for a Hugo in a particular category. A slate voter is someone who voted a slate of candidates mindlessly. That is, he/she voted for the whole slate and nothing but the slate. An organic voter is any voter who isn't a slate voter.

An organic nominee is one that didn't come from a slate, versus a slate nominee, which is one that did. 

The list of finalists is the top-6 nominees as chosen by any particular voting method. The organic finalist list the the list we would have had in the absence of any slates.

An organic finalist is an organic nominee that would have ended up in the top-6 list if there had been zero slate nominees. The #6 organic finalist refers the the 6th-ranked organic finalist. Ideally, the list of finalists would all be organic.

The slate refers to the list of six nominees that the slate organizer wanted to win in a particular category. (Or the 15 candidates, in the case of 3SV.)

subslate is a list of 5 slate nominees taken from a slate and given to a particular slate voter. Slaters need to use subslates to get optimal results because a ballot is limited to five nominees. Slate organizers can assign voters to subslates by means such as telling them to look at the last digit(s) of their telephone numbers. (E.g. for a 6-way subslate, find the last digit that's in the range 1-6 and that's your subslate.)

A ballot is what a member submits to WorldCon to make nominations for the Hugo.

Because subslates are smaller than slates, the number of slate votes that a slate nominee receives will be less than the total number of slate voters.

EPH points are computed from votes as a way to penalize slates. Just looking at the list of finalists (so not thinking about the intermediate process), any voter who had N of the finalists on his/her ballot contributed 1/N EPH points to each such finalist. A voter with only one finalist on his/he ballot gave one point and one vote to that finalist. A voter with five finalists on his/her ballot gave each one one vote but only 1/5 of a point. (With EPH+ it would be 1/9 of an EPH+ point.)

A.2.2 Traditional Method

To get an idea of how the general algorithm works, let's start with the old voting method, where every voter nominated up to 5 candidates, and the finalist list contained the 5 nominees with the most votes. We didn't compute that here, but understanding it will help a lot.

Imagine that we are making a slate and we start with zero slate voters. Obviously we don't get anywhere. Start increasing the number of slate voters, one by one. At what point do we get our first finalist?

Look at all the #5 organic finalists across all categories. Find the one with the fewest organic votes. As soon as we have that many slate voters, we'll tie that finalist and get a slate nominee into the finalist list. 

In general, just take the number of votes received by all finalists in all categories and sort them all into a single big list. That gives you the "jump points." When the number of slate voters reaches a jump point, the number of slate finalists increases by one. Fairly simple.

To compute stats for sweeps, do the same thing, but only use the #1 finalists rather than all finalists. This works because when the number of slate voters exceeds the number of organic votes that a #1 finalist received, that category will only have finalists from the slate; there will be no organic finalists.

That's all there is to it. All the other voting schemes are attempts to require more slate voters before giving up spots on the finalist list. Our task is to compute how many that is for each organic finalist across all categories. (That is, for each organic finalist in each category, we're computing how many total slate voters would be required to knock that finalist out of the list.)

Summary: For the traditional case, to knock a particular organic finalist out of the list requires slate voters > organic votes for that finalist. (Remember that, by definition, every slate nominee in every category gets the same number of votes.)

Clean and simple. It's also why slates were able to sweep so many categories; slate votes go up linearly with the number of slate voters, but organic votes only increase as the square root of the number of organic voters.

A.2.3 Standard 5 of 6

If the slate organizer only makes a single slate of 5, it will be impossible to sweep any categories, so the optimal strategy to attack the plain 5/6 system (that is, without EPH or 3SV) is for the organizer to make a slate of 6 nominees and then create 6 "subslates" of 5 by deleting one of the nominees from each. He/she then sends each slate voter one of the six subslates, evenly distributed.

This has the effect of diluting the slate votes for each nominee by a factor of 5/6. In order for a slate nominee to knock out any particular organic finalist, there must be 20% more slate voters (6/5) than there were organic votes for that finalist.

If the organizer knows in in advance that there are certain categories he/she can't easily sweep, then it would make sense to offer slates of just 5 candidates for those categories. Our numbers here do not reflect that strategy.

Summary: To eliminate any given organic finalist requries number of slate voters > organic votes for that finalist times 6/5

A.2.4 E Pluribus Hugo

As with 5/6, the optimum strategy to attack EPH is to create 6 subslates of 5 nominees each. Otherwise sweeping a category is impossible for the same reasons as above. As we will see, unlike 5/6, there is no advantage to the organizer creating a 5-nominee slate for categories that can't be swept, unless the organizer believes the slate will struggle to get even a single finalist in that category.

There are two cases here: beating the #6 organic finalist and beating any other organic finalist.

Beating the #6 Organic Finalist

The way EPH works, taking the #6 spot means beating the #6 organic finalist in terms of votes, not EPH points. Since the subslates dilute the number of slate votes, this will require 20% more slate voters (6/5) than the finalist's tally of organic votes. In other words, the situation for the #6 finalists is exactly the same as for the case of 5/6 (above). If the slate organizer thinks it will be hard to get even one finalist, then he/she should give everyone a list of 5. (It never makes sense to send a list of just 1.)

Beating the #R Organic Finalist for R < 6

In order for a slate to beat the Rth-ranked organic finalist, its nominees have to have more EPH points than the organic finalist does. This is because the way EPH works, in each successive round of the algorithm, the number of points any nominee has never decreases. It either stays the same or goes up. (Or the nominee gets eliminated, but we're only talking about the ones that survived to become finalists.) If a slate nominee starts off with more EPH points than any organic finalist above #6, it will never be subject to an elimination test. That is, it will never be at the bottom of the list, so it never has to survive the comparison.

The organic finalist will also have a number of points that's less than the number of organic votes, but the deflator will be around 1.5 or less--virtually never as big as 2. We have estimated values for the organic deflators for all finalists in all but two categories for 2014-2016. Our article Slate Voting Analysis Using EPH Data: 2014-2016 explains in detail how this was calculated.

So all we need to do is compute how many slate voters it would take before every slate nominee in the finalist list would have more points than any particular organic finalist.

EPH hands out points based on how many ballots each nominee appeared on. If we want to capture spots 5 and 6, that means we'll have two nominees from the master slate in the finalist list, so you'd think EPH would give each one half as many points as it had votes. Then we'd need double the slate voters (plus we've still got that factor of 6/5 to apply). 

But because of the overlapping subslates, that's not quite right. The slate item we want to put into slot #5 appears on only 5 out of 6 ballots. The one for slot #6 also appears on only 5/6 ballots. They appear together on only 4/6 ballots. So the slate nominee for position #5 gets 1/6 as many points as there are slate voters from the ballots that it is on alone and it gets 2/6 as many from ballots it shares with whatever ends up in slot #6 for a total of 1/2. The factor of 5/6 disappears.

In fact, at all positions except for #6, it turns out that we need 7 - R times as many slate voters as there are organic points for the organic finalist at that level. This is exactly what would happen if members were allowed to put six nominees on each ballot.

To see this, satisfy yourself from the combinatorics of the problem that if there are N = 7 - R slated nominees in the finalist list, then all N of them appeared on (6-N)/6 of the ballots and - 1 of them were on (N-1)/6 of the ballots--and there are no other combinations. The number of points will then be


Summary: To knock organic finalist #6 off the list requires number of slate voters > organic votes times 6/5. 

To knock organic finalist #R (R < 6) off the list requires number of slate voters > organic votes times (7 - N) divided by that finalist's organic EPH deflator.

Note: It is barely possible for the number of slate voters required to take spot #5 to be less than what's required to take spot #6. This has no effect on the graphs above.

A.2.5 EPH Plus

EPH+ is like the original EPH algorithm, except instead of dividing by N you divide by 2N-1. That is, instead of 1, 2, 3, 4, 5 the sequence is 1, 3, 5, 7, 9. Otherwise the calculations are the same as for EPH, with the caveat that the result isn't as elegant.  Also, MidAmeriCon II didn't give data for EPH+, so we'll have to estimate the organic deflators as well.

Organic Deflators


To estimate the organic deflators for EPH+, we look at the organic estimate again. Notice that these are fairly small numbers--between 1.2 and 1.5 for the most part. This suggests that most voters' ballots were down to a single candidate by the end of the EPH process, and that most others were down to just two. Making that assumption, we can estimate the EPH+ deflator from the EPH deflator as follows:

Assuming that the EPH deflator is a combination of ballots with one finalist and ballots with two then


For some a between 0 and 1. We know D for any given organic finalist, so we can compute a


We can define the EPH+ deflator in terms of a as well.


So substituting the value for a above, we can estimate the EPH+ deflator from the EPH deflator.


This will work so far as our assumption is correct--that nearly all organic ballots have either 1 or 2 organic finalists on them.

Slate Deflators

Using the same logic we used for EPH, the case for a single slate finalist is the same: it needs 1.2 times as many slate voters as organic votes for organic finalist #6.

For the case where R < 6, as above there are N = 7 - R slate finalists and so the number of slate points as a fraction of total slate votes will be


We need this in terms of R = 7 - N and we need the reciprocal, which is:


Summary: To knock organic finalist #R off the list requires number of slate voters > the finalist's organic votes times 6(13 - 2R)(11-2R)/(67-12">R) further divided by that finalist's organic EPH deflator. 

This messy formula hides an interesting result. Where subslates hurt the slates against 5/6, and they are neutral against EPH (except for position #6), they actually help the slates (slightly) against EPH+. Making smaller subslates (with fewer than 5 nominees) helps further at the higher levels (that is, helps in managing a sweep) but at the expense of making it harder to get 1, 2, or 3 finalists. 

As above, we'll assume slate organizers can't predict the future, so we'll figure 6 subslates of 5 nominees as the optimal strategy against 5/6, EPH, and EPH+ all three.

A.2.6 Three-Stage Voting (3SV)

In this system, there is a middle stage of voting after nominating but before the final vote. This allows members to preemptively disqualify bad nominees. This is meant to allow people to remove things like pornographic nominees submitted in order to embarrass the awards. The first stage of voting is the same as always--people nominate 5 candidates. In the 3SV stage, however, everyone gets to see the top-15 (but not the number of votes--just the names) and can delete any or all of them. If enough members vote to delete an item, it cannot become a finalist.

For the optimal attack on 3SV, the slate organizer prepares a slate of 15, divides it into three, non-overlapping subslates of 5, and distributes those evenly to all slate voters.

This now operates the same as in the standard 5 of 6 model, except that votes get multiplied by 3, not 6/5. At the point where there are enough slate voters (divided by three) to overwhelm the organic finalist in spot #6, they will have taken over all the spots from #7 to #15. This strategy defeats 3SG by forcing the members to make a list of finalists with fewer than 6 nominees--possibly zero.

Summary: To kick any given organic finalist out of the list requires number of slate voters > the finalist's organic votes times 3. 

This assumes the slate organizer doesn't really care which of his/her 15 slate nominees ends up in the finalist list, just as long as 6 of them do.

A.2.7 EPH/3SV

When EPH is used to calculate the 3SV list, the slaters need 9 items in the list before they can get one into the top 6. That 10th item will be one of 4 submitted by his/her subslate. This means the slate has to beat the number of points (not votes) for nominee #6 and the slate EPH deflator will be 4. It will be 4 for nominees 5 and 4, and will be 5 for nominees 3, 2, and 1. 

Summary: To kick an organic finalist out of position #4, 5, or 6 requires number of slate voters > the finalist's organic votes times 12 divided by the finalist's EPH deflator. 

To kick an organic finalist out of position #1, 2, or 3, requires number of slate voters > the finalist's organic votes times 15 divided by the finalist's EPH+ deflator.

A.2.8 EPH+/3SV

We use the same method as for EPH/3SV but with the EPH+ organic deflator and the EPH+ slate deflators.

Summary: To kick an organic finalist out of position #4, 5, or 6 requires number of slate voters > the finalist's organic votes times 21 divided by the finalist's EPH deflator. 

To kick an organic finalist out of position #1, 2, or 3, requires number of slate voters > the finalist's organic votes times 27 divided by the finalist's EPH+ deflator.

A.3 Slate Impact on Sweeps 

A slate sweeps an entire category if and only if it manages to beat the #1 nominee in that category. To compute this, simply use the same method as described above for finalists, but only take the #1 nominee from each category.

A.4 Raw Data

We've shared the Excel spreadsheet used to construct the graphs. The meanings of the columns should be obvious, given the explanations above, but, if not, feel free to post a question in the comments below.

2 comments (may contain spoilers):

  1. Thanks for all your work here. Good to get people considering all this.

    ReplyDelete
    Replies
    1. I just hope it helps people who're actually voting at WorldCon to make the right decision.

      Delete