LIZ GERARD: After the Telegraph’s polling fiasco can the public trust the pollsters?

After conducting an utterly unscientific opinion poll, I can exclusively reveal that 99.9% of them want it to stop. The outlier being Nigel Farage, whose entire reason for being would disappear along with that dread word.

Ah, but how do we want it to stop? To get on with it so we can think about something else? Or do we want it to be stopped: for Article 50 to be revoked? Two diametrically opposed viewpoints that would elicit the exact same “agree” response to a pollster who asked “Do you wish we could stop worrying about Brexit?”

For that question to have any meaning at all, it would have to sit among a lot of others to build a picture of what the person being polled really thinks. And how those questions are phrased, what order they are posed, and by what means – face-to-face, by telephone, online – all feed into the end result. The sample is obviously even more important: are those answering the questions self-selected, from a particular sector of society, of a particular age group? And how many of them were there?

The political polls most frequently published by newspapers deal with the simple question of which party electors would vote for if there were a general election tomorrow. Regardless of who is conducting the survey, the figures tend to be fairly consistent, so that a movement of more than a couple of points is regarded as significant.

But there are problems – pollsters are always getting it in the neck for failing to predict exactly how people will vote when they get to their little hessian cubicle. This is because it’s not just politicians who lie. Voters do too. Even when being questioned by a complete stranger they will never meet again, they want to be seen to be “good” people. So they’ll say they plan to vote Labour or Green because they say the health service or climate change matters most to them. And then they’ll give their cross to the Conservative candidate because he’s promised to cut their taxes.

Brexit is even more complicated. Remainers like to argue that the public mood is shifting towards staying in the EU and they cite countless opinion polls purporting to show that a majority agrees. Brexiters meanwhile claim that people are quite happy to leave without a deal. Indeed, the Daily Telegraph splashed last week on a poll that – it said – showed that a majority thought “Boris” should shut down Parliament to ensure we left by October 31.

Except it wasn’t a majority – it was 44% – and it wasn’t to get us out by Hallowe’en: it was “to do whatever was necessary” to prevent MPs “stopping” Brexit, which could mean at any time. The “majority” was achieved by excluding the “don’t knows”, which not only seems dishonest, but is also dangerous, since it could lull “your” side into a false sense of security.

Perhaps the Press regulator should demand the rigour of the Advertising Standards Authority – remember how “Eight out of ten cats prefer Whiskas” turned into “Eight out of ten cats whose owners expressed a preference…”

ComRes, the polling organisation responsible, defended the practice, saying that a “heady cocktail” of speculation, the approaching endgame and the silly season had encouraged “keyboard warriors” to pile in. Those “keyboard warriors” included polling experts such as former YouGov president Peter Kellner and Professor Will Jennings of Southampton University. They were, said ComRes chairman Andrew Hawkins, “brazen” in their eagerness to criticise a rival for doing what they routinely did themselves.

One example he gave was YouGov’s 25,000-strong survey for the People’s Vote organisation last December. Sure enough, in a commentary for The Observer, Kellner wrote:

“The biggest survey yet conducted on Brexit shows that Remain would comfortably win a referendum held today … if the choice is Remain versus the government’s withdrawal agreement, Remain leads by 26 points: 63% to 37%. If the choice is Remain versus leaving the EU without a deal, Remain wins by 16 points: 58% to 42%.” Guess what! He excluded the don’t knows. In fact only 46% backed Remain in each scenario.

So, too, did The Independent and its pollsters, BMG, when they claimed the country had reached tipping point in September 2017 with a majority now backing Remain. The survey, they said, mirrored the referendum result with 52% for remain and 48% wanting to leave. But the actual figures were 47% remain, 43% leave and 8% don’t knows. In these cases, voters were asked to choose between two options, whereas in the latest ComRes poll, people were asked to agree or disagree with a single statement, so the dynamics are slightly different. Also, Kellner’s gripe with the Telegraph was more bout the sequence of questions, which led almost inexorably to the “desired” response if interviewees were not to contradict themselves and make themselves feel stupid.

There are other ways that newspaper polls can deceive and delude. The polling organisation will publish all the data, but newspapers are never going to print every answer to every question. Especially when some of the findings are inconvenient.

That 44% figure has history when it comes to the Telegraph and ComRes. Last March, that same proportion agreed that Britain should leave without a deal if the EU failed to make further concessions. The Sunday Telegraph led with the headline “Public swinging behind no-deal”, which was accurate – 38% had felt that way in a previous survey – but still misleading. “Towards” instead of “behind” might have been more honest.

The poll did, indeed, find that people preferred leaving without a deal to Theresa May’s withdrawal agreement. But a true majority – 55% – said they would now vote to remain in the EU (which is not, of course, the same as saying that they wanted to abandon Brexit). That didn’t make it into print.

There is also the issue of the leading question and the choice of words. Do you want a second referendum? “Good heavens, no!” Do you want a people’s vote or a final say? “Yes please.”

Are polls designed to enlighten the world in general or to produce the answers the commissioner wants?

One former political editor with many years’ experience of working with polling organisations insists it is the former, and says there was always a lot of to-ing and fro-ing over the wording to ensure that the questions were robust and made sense. “Neither I nor the polling organisations would want to be associated with a slanted question that might affect our credibility.

“Moreover, there was a lively public commentary on the internet from informed analysts such as Mike 
Smithson and Anthony Wells, who quickly pointed out anything dodgy – as they still do.

“One-off polls, particularly in the current febrile climate, are different. The client is in the driving seat, since there are now so many polling organisations eager for business. But you cannot blame the pollsters, since most problems come from the presentation, which is in the hands of the media organisation, and papers can largely do what they want.”