A Conservative majority? Only a catastrophic exit poll error could put that in doubt now

So, despite a day of rumours and speculation the exit poll confirms the broad picture that the final opinion polls painted yesterday. The Conservatives are expected to win 368 seats, more than enough for Boris Johnson to push ahead with his Brexit deal and take Britain out of the EU at the end of January. For those seeking some signs there may, yet, be hope of a different outcome the question is always “Is the exit poll right?” But based on the estimate tonight even the largest expected errors would not be enough to change the outcome.

Few who study or work in polling will forget where they were when they saw the 2015 exit poll. After weeks of polling showing it was too close to call, anticipation of a tense election night—anxiously waiting to see who would be able to form a government in the following days—was high. But as Big Ben chimed 10pm, the exit poll had found the Conservatives on course for a majority. Famously, the late Paddy Ashdown said he would “eat his hat’” if the prediction that the Liberal Democrats won just 10 seats proved correct—he was of course eating his words if not his hat by the following morning. Fast forward to 2017 and a similar shock result was announced: after beginning the campaign with a seemingly unassailable lead, the polling had showed this narrowing, but it was still largely predicting a small Conservative majority. However, the Conservatives found that this time the 10pm bongs revealed a lost majority.

After delivering two accurate but surprising predictions in a row, the exit poll has taken on something of a mystical quality, expectations about its veracity are high. But can it deliver when collecting the data on a cold December day as well as it has done in the milder days of June? The statistician behind the current methodology is clear that this depends on many things, not least a little bit of luck.

How does it work? The exit poll is not a traditional poll, it does not aim to find a representative sample of the electorate and infer behaviour from them. It works because it makes use of things we know, the election results from last time and the demographic profiles of constituencies and combines this with reports from voters of their actual behaviour on polling day. In this way it does not have the same uncertainties around turnout, late swing and sampling that traditional polling methods have. Each person in the exit poll sample has cast their vote on polling day, there are no stay at home voters, undecideds or last-minute changes of heart to worry about. Though it does not sample those who have cast their votes by post prior to polling day this has not, so far, been a cause for concern.

By the standards of newer modelling techniques, the samples used are relatively modest. Between 100-200 voters in around 100 polling stations (individual polling stations not constituencies as a whole) are sampled. By comparing how these relatively small areas have changed with previous elections, changes across whole constituencies can be inferred. As with all such models there is a degree of uncertainty about the estimates, which gives a band of “likely” results around the precise estimate of seats for any given party. For the exit poll this is in the region of 15 seats, and so a Conservative majority looks very likely.

The current methodology of the poll was developed for the 2001 election and used fully for the first time in 2005 (prior to 2001 different broadcasters had different exit polls, often with notably large errors—for example predicting a hung parliament in 1992). Since 2005 it has had a remarkably good record in correctly calling the outcome even when traditional polling has failed. In 2005 and 2010 the exit poll got the winning party’s majority spot on and in 2017 was just four seats out on the Conservative total. Ironically, the exit poll in 2015 is remembered for accurately calling a Conservative majority but in fact had underestimated the Conservative seat total by 15 seats.

The 2015 election had many factors in play which were expected to make the call especially difficult: the rapid rise of the SNP, collapse of the Liberal Democrats and the emergence of Ukip as an electoral force. Many similar factors are also in play in 2019, with the added uncertainty of tactical voting and the impact of a December election. The final outcome is probably not in doubt. But in a few hours we will know for sure how accurate the exit poll has turned out to be. In the meantime, watching results and how they relate to the predictions of the exit poll should give us a good indication of the kind of night we are in for.