The polling industry has been hit hard by high-profile misfires in recent years. Pollsters got last November’s Kentucky governor’s race wrong, as they did the 2014 midterm elections. They have missed predictions in elections in Israel and Britain.  But exactly why the polls err often remains a mystery. Potential sources for error abound: The initial samples could be biased, the likely-voter models may not reflect the actual electorate, or voters could make last-minute decisions that make even an accurate poll wrong on Election Day. Pollsters have few tools for untangling these distinct problems.  But a new study by researchers at the Pew Research Center, pollsters themselves, offers new insights into those factors during the 2014 midterms, when polls failed to anticipate a huge Republican landslide. The study found that likely-voter screens may not have adequately anticipated the Republican turnout advantage, and that undecided voters might have broken toward the Republicans.  Pew’s analysis, on balance, is good news for the polling industry. The conventional wisdom holds that traditional polls have been less predictive because of plummeting response rates in an era of cellphones and caller ID. But there is relatively little evidence that the polls are less accurate than they used to be, or that declining response rates have hurt the accuracy of polls after weighting — a statistical technique to adjust data for underrepresented groups like black or young voters.  If all pollsters faced the issues observed in the Pew data, then it’s possible that the errors weren’t attributable to the inability to reach people, but are instead issues unique to election surveys that pollsters will never solve — but could improve on.  The analysis was conducted by matching the panelists from Pew’s American Trends Panel to a voter registration file. This gave Pew the ability to see which of its respondents ended up voting in 2014, and which did not. Panelists were contacted again after the election to see how many changed their vote from what they had told pollsters.  The initial Pew survey, conducted in September 2014, found Democrats leading Republicans nationally by four percentage points in House races among registered voters. But in the end, just 61 percent of respondents who supported Democrats actually voted, compared with 73 percent of Republicans. If one had been able to question only those people who actually voted, the survey in September would have shown Republicans leading by three points.  A traditional likely-voter screen, asking voters questions like whether they’ve thought about the election or whether they’ve voted before, would have shown a tied race, according to the Pew analysis.  In a postelection survey, the verified voters said they supported Republicans by six points, about the same as the G.O.P.’s margin of victory in the national popular vote for the House elections. Those voters who in September said they didn’t know whom they would vote for ultimately broke Republican by a 49-to-39 percent margin; those who said they would support another candidate ultimately broke Republican, 41 to 22 percent.  It is possible that postelection survey respondents were more Republican because of the tendency for voters to report voting for the winner by a greater margin than the actual results. But it is also possible that this reflects a genuine late shift toward the G.O.P., something that was suggested to me by several pollsters in the aftermath of the 2014 election.  Either way, the Pew data is consistent with the possibility that much or even all of the polling error could be accounted for by shifts in undecided voters or problematic likely-voter screens.  The Pew data raises questions for pollsters, particularly those commissioned by mainstream news media organizations. The Pew report considered a number of options for improving likely-voter models. But regardless of the exact technique, Pew found that the accuracy was always improved by incorporating voter history data from the voter registration file — something that few mainstream media polls currently do.  Most mainstream media pollsters call random telephone numbers, which aren’t attached to any additional data on their respondents. Pollsters could match these telephone numbers to the voter file, but they couldn’t match all of them — losing data from a large number of respondents.  There are good reasons mainstream media organizations have tended to prefer random-digit dialing. It provides a portrait of the entire adult population, not just voters — which may be important for many questions of public opinion other than estimating election results. Inertia is also a factor: Some traditional pollsters are relatively unfamiliar with voter files or have little capacity to handle and model very large data sets.  But pollsters working for political candidates have all but entirely moved to conducting their surveys straight off voter registration files. Past-voter history isn’t everything, but it is an extremely strong predictor of future turnout. Many people misreport their vote intentions and vote history, so asking people whether they’ve voted in recent elections isn’t an adequate replacement.  It’s hard to know how often polls have been wrong because of likely-voter screens in recent elections. But it certainly seems like a big contributor, given that self-reported vote intention is such a weak predictor of eventual turnout. It even seems possible that it has been a growing problem for pollsters: As Democrats have become increasingly dependent on irregular voters, it becomes more important for polls to accurately model the composition of the electorate.  Accurately capturing who votes could easily determine the success of pollsters during the primaries, when only a fraction of registered voters will participate. Donald Trump and Bernie Sanders are both drawing on irregular voters who have relatively little history of participating in primary elections. Just how many of their supporters actually vote could decide key states, along with whether the polls face another wave of misfires in high-profile elections.
Polls often err, but it can be hard to identify a cause. A new analysis by Pew offers some interesting guidance.