Take your pick. A Labour majority of 286, or merely 156? Either would deliver Keir Starmer a landslide. However, the 130-seat difference matters. Suppose Labour’s lead shrinks, and with a week to go, one poll shows it heading for a 120-seat majority, while another indicates a hung parliament. The political dovecotes would flutter furiously. Now, when things are a bit less frantic, is a better time to investigate the matter.
The 286 majority comes from Survation, which shows Labour leading for a much bigger victory than Tony Blair’s 179-seat majority in 1997. YouGov’s figures are just a shade short of that achievement.
What, then is going on? Let’s start with the similarities. Both were large scale surveys. Survation’s sample was 15,029, YouGov’s 18,761. (Normal surveys generally have samples of 1,000-2,000.) Both used a technique with the clumsy name of multilevel regression with post-stratification. Large-sample MRP surveys enable pollsters to look in detail at men and women, young and old, rich and poor, Leave and Remain voters, homeowners and tenants and so on. They then match this data to the profile of each constituency. When YouGov produced its surprising but accurate prediction in 2017 that Labour could win Canterbury, it was because it detected a sharp swing to Labour among students, and Canterbury had lots of them.
Another similarity is that both surveys report similar Labour leads in national voting intention: Survation 19 per cent, YouGov 17 per cent. If they use the same MRP method, they should produce similar figures for seats won.
A traditional, non-MRP, uniform swing projection, adjusted for tactical voting and Scotland’s distinct voting trends, would convert a 17-point lead into a Labour majority of around 120, and a 19-point lead into a majority of around 140. YouGov’s figures are a little more generous to Labour, while Survation’s look off the scale.
Is a 286-seat Labour majority remotely plausible? I have gone through Survation’s seat-by-seat data. Some of its projections are—how shall I put this?—surprising:
Overall, Survation predicts big changes in some 30 constituencies. Those changes include:
A 3 per cent plus swing to the Conservatives in East Ham, Hackney North & Stoke Newington, Hackney South & Shoreditch, Holborn & St Pancras, Liverpool Riverside, Liverpool West Derby, Manchester Rusholme, Peckham, Queen’s Park & Maida Vale and Tottenham.
It predicts Labour will win from a distant third in Didcot & Wantage, East Grinstead, Harpenden & Berkhamsted, Hazel Grove, Hinckley & Bosworth, South Cambridgeshire, Sutton & Cheam, Torbay, Woking and Wokingham.
And, finally, that the Liberal Democrats will slump to third place in Bicester & Woodcock, Chippenham, Chelmsford, Dorking & Horley, Farnham & Bordon, Henley & Thame, Mid Dorset & Poole, North Cornwall, North Norfolk and Thornbury & Yate.
Odd things can happen locally in general elections. It is just possible that one or two of those 30 Survation projections might come true. But none of them look plausible to me. YouGov does not detect any big swings to the Tories in Labour’s heartlands; and it disagrees with each of Survation’s projections for the 20 seats shown above that are being defended by the Conservatives.
To make sense of all this, let’s take the three groups of seats in turn.
First, seats projected to swing to the Conservatives. Underperforming Labour MPs include Keir Starmer in Holborn & St Pancras and David Lammy in Tottenham. Has Survation discovered that they put their constituents off? No, it’s just that all 10 seats in this list have massive Labour majorities. Survation’s model is designed to pull Labour’s vote down in its safest seats—and push the Tories up where they are weakest. Hence these swings to the Conservatives, despite Survation’s own figure of a 15.5 per cent swing to Labour nationally.
The second group reflects Survation’s estimates of the swings at the other end of the scale. If Labour is losing votes in its strongholds, then the only way we can get to a large national swing is if Labour makes massive gains where its support last time was weakest. According to Survation (but not YouGov), this is precisely what is happening. That is why, starting in third place with less than 20 per cent support, it surges not just past the incumbent Tories, but also the Liberal Democrats, who regard these as their target seats. YouGov shows Labour remaining in third place in all 10 seats, and the Lib Dems gaining six of them.
Third, Survation expects the same pattern—vote losses where they are strong, vote gains where they are weak—to apply to the Liberal Democrats. This is why they are shown winning just 22 seats, fewer than any other pollster offering seat-by-seat projections. Not only does there appear to be no tactical voting, Survation’s model implies ANTI-tactical voting, with many Labour and Lib Dem supporters voting for the party LESS likely to defeat the Conservatives locally. This is not only implausible; it contradicts what has happened in recent byelections, where voters have plumped in large numbers for the anti-Tory candidate MOST likely to win. Once again, YouGov disagrees, not just in individual seats, but the underlying structure of the shifts in votes. It incorporates some tactical voting.
All in all, I believe that Survation’s methods are flawed. There is, though, a further problem, not just with this survey but with all models that convert national data into seat-by-seat forecasts.
This is because swings vary for all sorts of reasons. MRP picks up those that are based on national, or sometimes regional, demographic variations. Hence YouGov’s success with Canterbury in 2017: students across England were moving to Labour in large numbers.
However, some variations are more granular. To take just three possible examples: Tory middle-managers living in big cities may behave differently from those in villages and market towns. Likewise retired Labour folk who live near the coast compared with those in its traditional industrial heartlands; and Lib Dem parents who commute by bus or train compared with those who drive to work. To get at these, and many other, kinds of fine-grained variations, much bigger national MRP samples are needed.
But even a huge sample is at risk of missing variations that are specific to a single seat, or a small cluster of them. Local controversies—such as over transport, housing, hospitals, crime, the local council—can make a difference. In some places the quality of local candidates might matter (including, very occasionally, the reputations of incumbent MPs seeking re-election). Such variations need not be large to have a material impact. Without firm local information, predicting closely fought constituencies from even the largest national sample is hazardous.
What a well-designed large-scale survey can do is give us a good idea of how many seats will change hands at an election, and which party will win the great majority of them. But local variations that take place under the MRP radar mean that some seats unexpectedly change hands on above-trend swings, while others don’t because they have below-trend swings. In a good MRP survey, these two opposite errors will balance out and ensure that the overall state of the parties is accurate.
Taking everything into account, the key issue for any seat-projection survey is: does it get the broad pattern of the swings between the parties right when it estimates the outcomes in different kinds of seat? Survation’s methods look too simplistic. What about its rivals? This raises deeper questions about the basic nature of MRP. I shall return to these in due course.
