(Warning: This is a long post about a mostly theoretical problem with the Oscars selection process. Viewer discretion advised.)
The Oscars are now only about a month away and the nominations are out, with 9 films nominated for Best Picture rather than 10. Being far more an elections nerd than film nerd, the change in the rules that allows this is more interesting to me than the actual results. The view of the academy that led to the change is that it’s not necessarily the only job of the nomination process to do a good job of aggregating the preferences of it’s members but also that any film nominated should have some claim to be an outstanding film worthy of the honour (and in any given year there might be a different number of films that qualify). Given this we get a new Rule 2 for the Best Picture Nominations:
The pictures receiving the highest number of votes shall become the nominations for final voting for the Best Picture award. [a] There may not be more than ten nor fewer than five nominations; however, [b] no picture shall be nominated that receives less than five percent of the total votes cast.
Now this rule is really the reason for the post, because I’m not convinced that the system they use is capable of ensuring both a and b – but it’s going to take a while to get there.
In the first place a system that can have a variable number of winners poses a bit of a problem for designing an electoral system as typically these are mechanisms for distributing limited goods (seats, awards, etc) where the number of these goods is fixed – you may not know before the election who’s going to win, but you do at least tend to know how much winners there going to be. Before we look at exactly how they’ve achieved this it’s helpful to look at how the old system works because it’s been cannibalized to produce the new process. I’m building on Steve Pond excellent series of posts on how Oscar votes are counted (in both old and new methods), but I’m going to restate of what he said because some of his terminology is a little off and to be more precise about the process (which may or may not be the same thing as being clearer). Basically Best Picture nominations used to be decided on a slight variation on Single Transferable Vote.
The Old Method
- Gather all ballots
- Eliminate films with less than 1 first preference vote.
- Calculate Droop Quota for number of seats remaining (start at 10).
- Sort votes by highest valid preference
- Elect any candidates above the quota.
- If a candidate’s votes are more than 20% above quota, revalue all votes with a new value of surplus/total x current value. If a candidate is between 100% and 120% of quota, discard votes.
- If no candidates have been elected, eliminate the candidate with fewest votes.
- Repeat (3 through 8) until all seats are filled
The only atypical bits are Steps 2 and 6. Step 6 requires an extra 20% beyond the quota before redistributing votes, which I’m assuming this is a practical measure so they can count by hand and not have to bother moving hundreds of votes around with very small values, while Step 2 is a weak version of a requirement for initial support that’s the main purpose of the new system. Otherwise this is a pretty typical STV process, I just write it out because of how similar the new method is:
The New Method
- Gather all ballots
- Calculate Droop Quota for number of seats remaining (start at 10).
- Sort votes by highest valid preferences
- Elect any candidates above the quota
- If a candidate’s votes are more than 20% above quota, redistribute all votes with a new value of surplus/total. If a candidate is between 100% and 120% of quota, discard votes.
- Eliminate any candidate with less than 1% of the vote and redistribute to their next highest valid preference.
- Eliminate any candidates with less than 5% of the original vote total (including votes spent through electing a candidate).
- If Number of remaining candidates (including elected candidates) <= 10 then
- Elect all remaining candidates.
- If Number of Candidates > 10 then
- Calculate Droop Quota for 10 seats
- Sort votes by highest valid preference
- Elect any candidates above the quota
- If a candidate’s votes are more than 20% above quota, revalue all votes with a new value of surplus/total. If a candidate is between 100% and 120% of quota, discard votes.
- If no candidates have been elected, eliminate the candidate with fewest votes
- Re-calculate Quota.
- Repeat (9.a through 9.f) until all seats are filled
If putting it like that is confusing, John Young’s description might be a better read.
Now this process is on the face of it a bit odd, it’s a hacked-open version of STV that retains a quota but has an escape halfway through – it runs one round of STV, then abruptly eliminates a number of candidates. The old requirement on at least one first preference has been beefed up and moved further down the process - why not discard candidates with less than 5% of prefs right at the beginning and only bother with any preferences if you end up with more than ten candidates left? As it turns out while it looks messy, the opening part of the process is quite important.
The goal here isn’t just that a film should have a fair amount of initial support but also that there must always be between 5 and 10 candidates. The first part is quite easy but as it requires setting an absolute standard for ‘initial support’, it makes it hard to guarantee that there will be at least 5 candidates. The problem can come from two directions: over-concentration – where four or less films receive so many of the first preferences that it’s impossible for any other film to reach 5%, and dilution – where the vote is spread over so many films that less than five reach 5% at all.
Over-concentration
Retaining the early stv-esque stages of the process is essential to preventing over-concentration. If a few films are locking up all the votes they’re going to be significantly over-quota and so keeping the quota and redistribute rules mean that their surplus votes will be spread around the other films, making it much more likely that more than five films will make it through. So while the official press release says a film must have 5% of first preferences this isn’t strictly true, a film’s 5% can come through a mixture of their own votes, redistributions from unpopular files and the surplus of very popular films. The take home point there is that second preferences might have been important even when less than ten films emerge, as the second preferences of front-runners or <1%ers might have tipped a few more over with their surplus. What isn’t clear is if they run steps 4 to 6 in a loop until all possible candidates have been elected before proceeding to the 5% rule, as steps 5 and 6 could push films above quota. If they’re not doing this looping, when say three films dominate 90% of the first preferences, and those three all give their second preferences to a fourth film then you’d still end up with less than five films. This isn’t necessarily a ridiculous example, as at that point any preferences for films 1-3 don’t need to specify 4 as a second preference to cause the problem, 4 simply has to be the highest valid preference once 1-3 have been removed through election.
Dilution
From the other end is dilution, which is more unlikely but not at all negated by the system. If we imagine a situation the first preferences are evenly split between 21 films then all are above 1% so no redistribution takes place and none are above 5% and there are no winners at all. This is obviously an extreme outcome and it’s really unlikely that voters will be that split on their film selection – however the system itself creates voting incentives that could lead to the vote being more spread out than voters’ actual preferences might suggest. While Pond thinks that most Oscar voters aren’t clued up enough about the system to strategize their vote (and he’s probably right) for the purposes of exploring the system a bit, I’m going to go into ways this system could be played.
The 5% threshold is around half the initial quota that a front-running film would have to clear in the first round, so in this sense for a film to even reach the quota in the first round means that half its votes are wasted as they do nothing to change the result. If you’re a voter that has several films you want to be nominated (say two, Film A that’s a shoe-in and you like more, and Film B that’s chances are more dubious) and you want to maximize the value of your vote, listing your preferences honestly 1st and 2nd isn’t the way to go. If there are 1000 votes, and 125 voters vote A>B (these are the only voters who have preferences for these films) then A will be nominated and 25 votes will be redistributed to B – this falls short of 5% and B will not be nominated. However, as A can be nominated with 50 votes, then those same voters can split their first preferences 75 for A, 75 for B and they’d both be nominated. In real life if you were one of those voters you’re not really able to coordinate with all the others on the optimal distribution of your votes, which means your best course of action is to put B as your first choice and free ride on the other voters to elect A anyway. The trouble then comes is enough people do this for B (and C, D, E, etc) then even an obvious and wide-supported nominee like A might fail to make 5%.
Now I think this is an unlikely outcome but given a clued up electorate it is at least possible that only four films might make it through. That possibility makes Rule 2 flawed as the requirement for 5% support is in conflict with the requirement for at least five nominees. Strictly speaking the rule should prioritize one requirement over the other, either making the 5% rule absolute (allowing less than five nominees to emerge) or the nominee number absolute (requiring that the old rules kick back into play if there are less than five over 5% as well as if there are over ten.)
Better methods?
So given these (largely abstract) problems, is there a better way to count? In terms of improvements to this system, the looping I talked about earlier should definitely be used if it’s not already and in step 2, a quota of 5% should be used instead of the droop quota as if even two films get in through quota in the first round that’s 10% of the votes unnecessarily removed from circulation, which only increases the risk of dilution. In terms of doing something completely different it’s worth thinking about what we’re actually trying to accomplish here. STV is a good way of handing elections with many candidates for many positions but the problem in this case is it can produce compromise winners, who are few people’s ‘great!’ choice but many people’s ‘alright’ choice – what we want is a way of discriminating against these winners and using first-preferences is a good way to that (and is used in other STV systems, for breaking ties in the ERS97 rules for example). As we’ve seen these rules aren’t doing that cleanly, as many second-preferences can also come into play in this ‘initial support’, is there a better way to do this?
My solution would be to run the old rules to produce 10 nominees – but then to remove any nominees who are not in the top 10 of first preferences received. This seems to me accomplish the same basic goal (variable number of winners based on initial support) in a more straightforward way. To ensure that at least five nominees survive, the exact rule should be that ‘Nominees should be checked to see if they should be eliminated in inverse election order (and then within rounds, the nominee with smaller surplus checked first) for a maximum of five eliminations.’ My gut instinct says it’d be more likely to be around the high end of 8-10 rather than the low end but given that this year 9 nominees came out, that’s probably not that different in most cases and as it has the possibility of chucking candidates out and so accomplishes the same basic purpose.
In short, Rule 2 potentially contradictory, new process weird but accomplishes a purpose, might be better ways to accomplish that purpose, Oscars aren’t made of solid gold, and Oscar himself stands on a film reel which one day will be as wonderfully anachronistic as the floppy disc you click when you hit ‘save’. That about covers everything I think.






