The recent failures of electoral polling have been over-stated. Our era is politically volatile, but not unusually so. This is the second post in our series exploring uncertainty and the challenges it poses to social science.
Shocks, spills and surprises are all the political rage these days. The UK’s vote for Brexit, the election of Donald Trump as President of the USA, and now a UK general election that saw the Conservatives squander a commanding polling lead, have led everyone at all interested in the public sphere to ask an entirely rational question: is there any point trying to forecast anything anymore?
This looks a little overdone from a historian’s point of view. Uncertainty is a simple fact of political, economic and social life, but over the long term human history has undoubtedly been far more volatile than it is now. As significant as these events are, a few surprising national votes, some populist revolts against apparently distant governing classes, and some startling breaks with tradition (think of the collapse of the French Socialist Party in this year’s elections) do not make for a fundamental caesura in political life. Commentators focused on a range of possibilities and probabilities, rather than simple ‘yes’ and ‘no’ answers, have always understood this: the statistician Nate Silver, for instance, began his coverage of the recent General Election with the words ‘the UK snap election is riskier than it seems’.
The very rules by which politics is played can change very rapidly, and indeed always have. The Irish famine of the 1840s, and the Tory split that it helped to bring about over the repeal of the protectionist Corn Laws, were very hard to predict. The Liberal implosion over Irish Home Rule could have taken many different paths had Prime Minister William Gladstone and his opponents chosen different strategies. In inter-war Britain, a cross-class party dedicated to a governing ethos (the Liberal Party) was displaced as the main left-wing party by an urban and unashamedly class-based Labour Party, making three-way general elections very volatile and hard to call in a first-past-the-post voting system. Very few expected the Exchange Rate Mechanism crisis of the autumn of 1992 to blow up as quickly as it did, nor that the Conservatives’ reputation for economic competence would be shattered for a generation.
We may be experiencing a similar period of changing rules, but nevertheless, political prediction itself does not actually seem to have deteriorated markedly, even taken in the narrow sense. As an example, we might look at the performance of UK opinion polls over the last few years. The final polling average, if we take the last survey conducted by each pollster in the field, was 43.6% for the Conservatives, and 36% for Labour – a Conservative lead of 7.6%. The actual result in Great Britain (excluding Northern Ireland) was 43.5% for the Conservatives and 41% for Labour. So, relatively unusually, the polls had underestimated Labour – although they were quite accurate in terms of the Conservative score. This meant that, instead of a Conservative lead of 7.6%, the party’s advantage was only 2.5% ; that is, a polling ‘miss’ on the gap between the two main parties of 5.1%. On the face of it, that is not a great result for the pollsters and forecasters. But it is important to realise that this is not a very bad performance by historical standards. As Silver’s figures on FiveThirtyEight.com show, the same ‘miss’ was 5.9% in 2015, 4.8% in 2001, 4.7% in 1997, 9.1% in 1992, 3.6% in 1987 and 5.1% in 1983.
So what has happened to make us think that we suddenly live in a world of unique uncertainty? The first important signal that we appear to be mistaking concerns the results of the elections listed above. If we listened to the conventional wisdom of newspaper commentators (who certainly have a far worse record than pollsters), the 2010 result seemed destined to be much stronger for the Conservatives than the actual outcome. The race looked, on the other hand, likely to be very close in 2015, with perhaps a lean towards Labour, an expectation not borne out by David Cameron’s surprising majority. In 2017, almost everyone started the campaign thinking that Labour were going to be routed. They increased their vote share, entirely unexpectedly, by nearly ten percentage points. As such, we have had three surprises in a row in terms of the headline result, and that strong binary signal from the world of ‘win and lose’ is blanking out previous polling misses. We need a much more gradated view of polling successes and failures. Everyone expected the Conservatives to win in 1983 and 1987, and they did. Everyone ‘knew’ that New Labour would be elected and returned in 1997 and 2001, and they were. It is very easy to overlook the medium-sized polling errors in those years if you just focus on the winning party.
The second thing that has happened is that the UK held two elections – in 2005 and 2010 – where the polls did pretty well. In those years, the average polling error was ‘only’ 3.3% and 0.7%: a break in UK polling’s lukewarm record that perhaps lulled us into thinking that adjusting for ‘shy Tories’, and changing some sampling methodologies, had made the industry’s problems go away. They obviously had not, though the polling firm YouGov’s remarkable success with its Multilevel Regression and Post-Stratification model in 2017, and Survation’s superb record in 2015 and 2017 (even if they unwisely declined to publish the former set of eve-of-poll results) do promise more and better breakthroughs in the future.
Our crystal balls are not broken. They might be cracked – they are certainly cloudy. But this is no time to give up our attempts to understand the future, and to learn from past experience. Indeed, we often defend economic and social scientific methods precisely because they are falsifiable. Some polling methods have accordingly proven to be inaccurate, but now is the moment to redouble our efforts to understand our failed projections as well as our successes. Our world has not suddenly become uniquely uncertain. It has always been unpredictable. The fact that politics is still capable of surprising us does not mean it is chaotic. We will just have to work harder and retool ourselves, as we seek to trace out the complex and evolving rules domestic and global politics now operate under. It could be an exciting prospect.
This article is part of a new SPERI Comment series on researching uncertainty. Read all of the articles in the series so far here.