Polls are Bollocks, Apparently

This is an article that anybody who watches, reads or otherwise consumes election news should read. The Dirty Secrets of Pollsters describes the current shameful state of election polling in Canada. The article states that the refusal rate for polling phone calls stands at about 80%. That is, for every 10 calls the polling company makes, they get 2 successful survey results.

Don’t they imagine that this is skewing results somewhat? I mean, when only 20% of the population is willing to respond, doesn’t that suggest that you’re going to get a certain type of person? That the poll isn’t even remotely indicative of the population as a whole? Media outlets don’t cite this when promote polls. As the article notes, “The margin-of-error statements in news reports of polls are based on an assumption that the people interviewed make up a random sample; that is, everyone has an equal chance of being interviewed. That may not be as true today as it once was.” Clearly not.

A spokesperson from Ipso-Reid defends the polls, saying that they’ve predicted every election outcome for the past 25 years. That’s a dubious statement in itself–every federal election or provincial and municipal too? Have they merely been able to pick the winner correctly? It’s hardly a remarkable achievement to guess who’s going to win federally five or six times.

Two other worrying points from the article follow:

Reid [formerly of Angus Reid] was also critical of how the media report poll results. Reporters often highlight small shifts in poll results, he said � changes that could be due to nothing more than the fact that numbers will always tend to bounce up and down from poll to poll within the margin of error.

“You’ve got the Globe and Mail looking at a drop of four points or three points and saying this is a major drop,” he said. “For all we know, within the normal margin of error, there was no change.”

A standard national poll will usually be based on about 1,000 interviews from coast to coast, which means that between 100 and 150 interviews will be conducted in a province the size of B.C. What the media often don’t explain, however, is that the results for B.C. will therefore be much less reliable than the national results.

Smaller samples mean larger margins of error and a sample of 100 carries a margin of error of plus or minus 10 percentage points, 19 times out of twenty. That means that if the “true” support for the Conservatives in B.C. is, say, 45 per cent, you can expect a well-conducted poll based on 100 interviews to give you a result for the Conservatives anywhere between 35 per cent and 55 per cent. Not quite as precise as the national results.

That’s simply ridiculous, and shameful that the media foists numbers on us that have such a loose connection with reality. I’m disregarding all election polls from here on in–they’re clearly worthless.