Read more: Can Sadiq Khan keep London safe?
One clear winner emerged from the May elections. Curiously, the media managed to ignore the victory. In stark contrast to the aftermath of last year’s general election, when everyone attacked the performance of the opinion polls, I can find no journalist or politician this time praising the pollsters for getting London, Scotland and Wales right.
This does not mean that they achieved perfection. But, if the polls were judged like films or TV shows, they would deserve four stars out of five.
They all predicted Sadiq Khan’s comfortable victory in London. Four out of five polling companies were within one point of his final vote share of 57 per cent.
They all predicted that the Scottish National Party would slip back from its vote in last year’s general election, but still come out well ahead of its rivals. Three of the five polls showed the Conservatives overtaking Labour to become the official opposition at Holyrood.
YouGov, the only company to poll regularly in Wales, rightly predicted a slight fall in Labour’s vote, and Ukip’s breakthrough in the Welsh Assembly.
Now to the remaining doubts. The polls, especially those conducted online, still tend to understate Conservative support and overstate either Labour or Ukip support. In Scotland’s regional vote (the one that determined how many seats each party would win), the polls put the Tories on 18 to 20 per cent; they won 23 per cent. In the first round of London’s mayoral vote, four of the five polls overstated Khan’s round-one lead over Zac Goldsmith. In London and Wales, polls overstated Ukip five times out of six.
In short, the defects identified a year ago seem not to have been eliminated. This holds lessons for the coming referendum. I believe the recent performance of the polls casts light on one of the mysteries of recent months: the discrepancy between online and telephone surveys about UK membership of the European Union. If we take monthly averages—and so iron out random fluctuations in individual polls—then online surveys have put the race neck-and-neck in every month this year. In contrast, each month’s telephone polls have shown a clear lead for staying in the EU.
Two reasons have been advanced for this discrepancy. They are not mutually exclusive: both may be right. First, online panels contain too many people who have strong political views, and that in the case of the EU, this skews the results towards supporters of Brexit. This (so the argument runs) is because voters with the least interest in politics, and least likely to join online panels, are the ones who will end up preferring the safer, status quo option, and mostly vote to remain in the EU. Like all online companies, YouGov—my old company—goes to great lengths to avoid this bias; the issue is whether they have done so completely. Perhaps recent telephone polls have reached more of the voters who are least interested in the referendum debate and most risk-averse.
The second possible reason is that some people who veer towards Brexit may be more willing to divulge their true feelings in an anonymous survey than to a stranger on the phone. There’s a fair bit of evidence that online surveys are better than telephone or face-to-face surveys at eliciting honest answers on sex, drugs and tax-and-spend. (Online respondents are more likely to admit that they prefer more money in their pockets than better-funded public services.) Perhaps some people are reluctant to reveal over the phone that they plan to vote for Brexit.
The performance of the polls in the recent elections is not totally conclusive, but it suggests to me that the main continuing problem is to do with sampling. (If the greater problem were candour, online polls this time would have been more accurate than they were.) The polls, especially those conducted online, seem still to reach too few of the “silent minority”: those who take little interest in politics, don’t join panels and, sometimes, elude telephone pollsters too.
Normally this doesn’t matter much. To repeat: the polls this time got all the big stories right. But sometimes it does matter. I believe it is why online polls all understated support for first-past-the-post in the alternative vote referendum in 2011 and why most of them have overstated Ukip support in European Parliament elections—and why they misjudged the state of the Tory/Labour race in London and Scotland this year, albeit only modestly. In the case of last year’s general election, neither online nor telephone surveys reached enough poll-averse Tories.
If this analysis is right, then the polls, especially online polls, continue to have sampling problems when it comes to issues such as EU membership. I suspect that “remain” continues to enjoy a modest lead in the EU referendum campaign. Am I certain? No. Is a pro-EU victory in the bag? No—even if we look only at telephone polls, the race is too close to be sure of who will win. My guess is that, in the absence of some eruption, such as a new crisis involving refugees, terrorism or the eurozone, most late-deciding voters will opt for the status quo and vote “remain.” If so, there will be a fairly comfortable majority for staying in the EU. But if some kind of crisis does erupt, all bets will be off.
One clear winner emerged from the May elections. Curiously, the media managed to ignore the victory. In stark contrast to the aftermath of last year’s general election, when everyone attacked the performance of the opinion polls, I can find no journalist or politician this time praising the pollsters for getting London, Scotland and Wales right.
This does not mean that they achieved perfection. But, if the polls were judged like films or TV shows, they would deserve four stars out of five.
They all predicted Sadiq Khan’s comfortable victory in London. Four out of five polling companies were within one point of his final vote share of 57 per cent.
They all predicted that the Scottish National Party would slip back from its vote in last year’s general election, but still come out well ahead of its rivals. Three of the five polls showed the Conservatives overtaking Labour to become the official opposition at Holyrood.
YouGov, the only company to poll regularly in Wales, rightly predicted a slight fall in Labour’s vote, and Ukip’s breakthrough in the Welsh Assembly.
Now to the remaining doubts. The polls, especially those conducted online, still tend to understate Conservative support and overstate either Labour or Ukip support. In Scotland’s regional vote (the one that determined how many seats each party would win), the polls put the Tories on 18 to 20 per cent; they won 23 per cent. In the first round of London’s mayoral vote, four of the five polls overstated Khan’s round-one lead over Zac Goldsmith. In London and Wales, polls overstated Ukip five times out of six.
In short, the defects identified a year ago seem not to have been eliminated. This holds lessons for the coming referendum. I believe the recent performance of the polls casts light on one of the mysteries of recent months: the discrepancy between online and telephone surveys about UK membership of the European Union. If we take monthly averages—and so iron out random fluctuations in individual polls—then online surveys have put the race neck-and-neck in every month this year. In contrast, each month’s telephone polls have shown a clear lead for staying in the EU.
Two reasons have been advanced for this discrepancy. They are not mutually exclusive: both may be right. First, online panels contain too many people who have strong political views, and that in the case of the EU, this skews the results towards supporters of Brexit. This (so the argument runs) is because voters with the least interest in politics, and least likely to join online panels, are the ones who will end up preferring the safer, status quo option, and mostly vote to remain in the EU. Like all online companies, YouGov—my old company—goes to great lengths to avoid this bias; the issue is whether they have done so completely. Perhaps recent telephone polls have reached more of the voters who are least interested in the referendum debate and most risk-averse.
The second possible reason is that some people who veer towards Brexit may be more willing to divulge their true feelings in an anonymous survey than to a stranger on the phone. There’s a fair bit of evidence that online surveys are better than telephone or face-to-face surveys at eliciting honest answers on sex, drugs and tax-and-spend. (Online respondents are more likely to admit that they prefer more money in their pockets than better-funded public services.) Perhaps some people are reluctant to reveal over the phone that they plan to vote for Brexit.
The performance of the polls in the recent elections is not totally conclusive, but it suggests to me that the main continuing problem is to do with sampling. (If the greater problem were candour, online polls this time would have been more accurate than they were.) The polls, especially those conducted online, seem still to reach too few of the “silent minority”: those who take little interest in politics, don’t join panels and, sometimes, elude telephone pollsters too.
Normally this doesn’t matter much. To repeat: the polls this time got all the big stories right. But sometimes it does matter. I believe it is why online polls all understated support for first-past-the-post in the alternative vote referendum in 2011 and why most of them have overstated Ukip support in European Parliament elections—and why they misjudged the state of the Tory/Labour race in London and Scotland this year, albeit only modestly. In the case of last year’s general election, neither online nor telephone surveys reached enough poll-averse Tories.
If this analysis is right, then the polls, especially online polls, continue to have sampling problems when it comes to issues such as EU membership. I suspect that “remain” continues to enjoy a modest lead in the EU referendum campaign. Am I certain? No. Is a pro-EU victory in the bag? No—even if we look only at telephone polls, the race is too close to be sure of who will win. My guess is that, in the absence of some eruption, such as a new crisis involving refugees, terrorism or the eurozone, most late-deciding voters will opt for the status quo and vote “remain.” If so, there will be a fairly comfortable majority for staying in the EU. But if some kind of crisis does erupt, all bets will be off.
