What does the polling say about a possible Progressive Alliance?
Last month a story did the rounds about how a poll shows that if Labour, Greens and the Liberal Democrats did a deal to stand only one candidate against the Conservatives, Boris Johnson would be out of power. Answering that question is important in its own right, but Im also going to show the steps to the answer  showing how to approach evaluating an opinion poll finding that is in the news. (More on all that in my book, Bad News: what the headlines dont tell us.) The poll was conducted Find Out Now. They are a member of the British Polling Council, the self-regulatory body for British political poling and the full data tables are available online. So far, so good. The right methodology for the task  The poll was an MRP poll, which allows constituency level results to be extrapolated from large national samples. Thats a good methodology to use to answer questions like how would a seat pact work? as you need to get into how results might vary between different types of seats. But MRP polls should come with a big caveat. Basic opinion polling is a fairly widespread skill, with plenty of experts and long track records. MRP is not. MRP methodology is complicated, with only a small number of real experts and with a limited track record. It is also much more of a black box than more traditional polling. All of this means the only real judgement of how good a particular teams MRP model is comes when you have actual results to compare them against. Pollster track record  Find Out Now teamed up with Electoral Calculus for this MRP poll and there isnt much of an MRP track record for them. The 2019 general election prediction from Electoral Calculus was respectable, but it wasnt a pure MRP prediction. Rather, it included other data, such as polling averages. There was a MRP poll for the 2021 local elections but its accuracy is far from clear. The seat predictions were way off, e.g. predicting Labour to gain 280 seats when in reality Labour 326. However, the seat numbers were all rather strange and didnt seem to even add up to the right total number of seats (e.g. see the replies in this Twitter thread). Electoral Calculuss own published assessment of this prediction simply leaves out the seat numbers and only talks about council control and vote share. On those two measures, it did well. But on seat numbers, something went wrong. The actual questions asked  Turning to the poll specifically, it was based on working out a possible set of seat deals, picking one of Labour, Green and Lib Dem to stand in each seat and asking people in England and Wales (though not Scotland) how they would vote. People were given the choice of candidates appropriate for their own seat:  Q2. Suppose at the next general election that all the usual political parties are standing in your seat except that the [OTHER PARTY1] and [OTHER PARTY2] have agreed not to stand and are asking their supporters to vote [SELECTED PARTY]. [SELECTED PARTY, OTHER PARTY1, OTHER PARTY2 are Labour, Liberal Democrats, Green party depending on respondents seat]  The results were then fed into an MRP calculation to provide seat numbers. These were compared with seat numbers without a pact, apparently showing a big boost for having such a pact with Labour wining 36 seats more with a pact, the Lib Dems winning 14 more and the Greens 8 more. Those 58 gains remove the Conservative majority. But the with/without pact comparison was not a like for like one. The poll didnt ask a normal voting intention question and then a pact question. Rather, it just asked the pact question and compared the results from that with the results from a poll of polls, looking at standard national voting intentions. So although the MRP pact poll finds a difference from traditional voting intention polls, we dont know how much of that difference is due to the former being an MRP poll and how much due to the theoretical pact. If both MRP and other polls are perfect (or equally in error!) then that doesnt matter. But weve no way of knowing if that is the case, which means the supposed impact of the seat pact is left uncertain. Its a real shame that the MRP poll didnt ask both voting intention questions and so give a much better like-for-like comparison. Theres a risk that people may not predict their behaviour accurately. Theres also a risk that by just asking about X a poll question focuses peoples attention on it much more than you would get in reality. Take the YouGov poll from 2016 which I reported as New poll shows Lib Dems could overtake Labour, Ukip if politics polarises over Brexit. We got an election in 2019 that was rather like some of the scenarios in that poll but a result that was very much not like its findings. The poll was right in the sense of showing that Brexit might fuel a powerful Lib Dem performance. Might rather than would because other things can  and did  happen. So even if the MRP poll on seat pacts was perfect, it would still suffer from this forecasting problem. Those seat deals in detail  Before concluding what we can learn from this poll, theres another detail that needs examining. The poll was based on a theoretical set of seat deals. Some of those deals (see Appendix 2 in the report) were very unusual. Labour and the Liberal Democrats standing down for the Greens in this seat (!). Which highlights a problem: there are no Conservative-held seats in which the Greens are in a strong second place as the obvious challenger. So any sort of seat deal would be very difficult to agree, as it either involves other parties standing aside where they can legitimately think they have a better chance than the Greens, or it involves the Greens standing aside without getting much in return. The Conservatives wont do nothing  The Lewes example also shows one other risk. At 48% the Conservatives were not far short of an absolute majority in the seat anyway. The Conservatives would use it as a political weapon to win over support  e.g. by squeezing party Xs old support by saying a vote for X is really a vote for Y, and also be using it as a motivator for its own supporters  youve got to go vote to stop the other parties conspiring against us. A poll about a theory cant capture well the reality of the counter-reaction if theory were to become reality. The polls gives a clue that Green, Labour and Liberal Democrat supporters may be willing to switch to defeat the Conservatives. Its a tentative clue as it doesnt give a clear like-for-like comparison between seat deal and no seat deal. Its also a tentative clue as it doesnt capture the political dynamics and difficulties that would be involved in any such deal. Nor did it include Scotland. So its a clue that suggest theres a willingness to switch. But it doesnt go so far as to tell us the best way to make that sort of switching happen, i.e. it doesnt tell us enough to judge the relative merits of formal seat deals versus a more informal targeting of different campaigning and encouragement of tactical voting, 1990s-style. A piece of evidence, then, rather than an argument clincher.