European elections: the polling data has been good but the analysis has been terrible

It is tempting, but not quite true, to say that there are lies, damned lies and opinion polls. Polls, like other statistics, are often vital if we are to understand what is going on. But when they are twisted, then danger lurks.

With this week’s elections to the European Parliament, and the wider divisions in Britain over Brexit, high quality data and rigorous analysis are especially vital. The weekend polls provided examples of both good data and terrible analysis. They generated three near-certainties, one important doubt—and two giant whopping errors, one of them unwittingly amplified by Andrew Marr on his Sunday morning show.

Let’s start with the whoppers. The Sunday Expressfront page headline proclaimed: BRITAIN SAYS “LET FARAGE LEAD BREXIT.” The poll cited by the Expresssaid no such thing. ComRes asked people which of six party leaders would best at doing each of eight things. One of these was “leading the UK out of the Brexit crisis.” Farage came top with 28 per cent. However, the other five leaders together scored 72 per cent. More than twice as many people rejected Farage as backed him. But rather than acknowledge the fairly obvious truth, that public opinion is now badly fractured, the headline misled readers.

That is not all. The paper also reported: “The ComRes survey has also shown strong backing for no deal, with 63 per cent saying that the UK should just leave with no agreement if parliament rejects Theresa May’s deal again.”

This was picked up by Marr in his interview with Rory Stewart. Marr put it to the international development secretary: “A big, substantial majority of British voters now want no deal.”

That is simply not true. ComRes presented respondents with nine statements, and asked people whether they agreed or disagreed with each. One of these statements was: “If parliament will not agree to the Withdrawal Agreement on the table, then parliament has to accept No Deal as the consequence.”

The responses were: Agree 47 per cent, disagree 27 per cent, don’t know 25 per cent. The paper arrives at 63 per cent by ignoring the large number of don’t knows. To be sure, it is normal to exclude don’t knows when reporting voting intention, for the central purpose of such a question is to explore the distribution of votes among those naming a party. But with the ComRes/Expresspoll, removing don’t knows on agree/disagree questions is misleading—especially when, as in this case, the paper failed to report, let alone justify, this removal.

In any event, the question as asked does not justify the way it was summarised. It did NOT show either 63 per cent or 47 per cent “saying that the UK should leave with no agreement if parliament rejects Theresa May’s deal again.” ComRes did not ask people what “should” happen.

Further, it offered a false choice. A poll that set out to discover what people wanted in the event of the Withdrawal Agreement failing again would offer more options. Parliament could seek fresh Brexit negotiations under a new prime minister; or revoke Article 50; or call a general election; or legislate for a new referendum. ComRes wrongly offered a single alternative to the WA—and could still get less than half its sample to back it: a far cry from the Express’s 63 per cent or Andrew Marr’s “big, substantial majority.”

The three near certainties can be dealt with more swiftly. The three weekend polls, by YouGov and Opinium as well as ComRes, all spell trouble for the Conservatives (support ranging from 9-12 per cent), Change UK (3-5 per cent) and Ukip (2-3 per cent). Unless the last few days produce a dramatic turnround, the Conservatives will lose most of their 19 MEPs, while Ukip will lose all of theirs and Change UK will also emerge empty-handed.

This leaves us with the doubt—by which I mean the extent to which the three weekend polls diverge. As well as agreeing that the Tories, Ukip and Change UK are doing badly, they all show the Brexit Party well ahead. But they disagreed on which party is running second, and also on how the votes overall divide between Remain and Leave parties.

YouGov shows the Lib Dems in second place, with 17 per cent support, ahead of Labour (15 per cent). Opinium and ComRes both show Labour in second place, with 20-22 per cent, and the Lib Dems on 15 per cent. The seven-point range in Labour’s support is striking. It is not a one-off blip: YouGov’s recent polls have consistently shown Labour with significantly less support than other polling companies. Politically, the difference is significant: ComRes’s 22 per cent represents a modest three point dip since the last European elections five years ago; 15 per cent would be a catastrophic collapse.

A larger point concerns the balance of support for, collectively, the Remain and Leave parties. There are two ways to view this. One is to put support for Labour and the Conservatives to one side, and look only at the parties that are uncompromisingly Leave or Remain. These are the figures from the weekend polls.

Leave (Brexit Party plus Ukip): Opinium 36 per cent, YouGov 37 per cent, ComRes 33 per cent

Remain (Lib Dem, Green, Change UK, SNP, Plaid Cymru): Opinium 29 per cent, YouGov 36 per cent, ComRes 34 per cent

If Opinium is right, Leave-party voters outnumber Remain-party voters by a clear seven-point margin. The other two pollsters show them neck-and-neck.

The alternative way is to add Labour to the Remain side of the ledger, and Conservatives to the Leave side (on the grounds that this reflects the preferences of big majorities of supporters of both parties):

Remain: Opinium 49 per cent, YouGov 51 per cent, ComRes 56 per cent

Leave: Opinium 48 per cent, YouGov 46 per cent, ComRes 43 per cent

Take your pick: neck-and neck (Opinium), narrow 5 point Remain lead (YouGov), massive 13 point Remain lead (ComRes). The political consequences of each are very different.

In a week’s time, we shall know the numbers, and what they tell us both about pollsters’ accuracy and the public mood. The immediate prospects for British politics might become a little clearer. Then again, they might not.