How did pollsters get the Australian election result so wrong?
Some politicians and commentators have called for opinion polls to be scrapped following prime minister Scott Morrison’s shock win in the Australian federal election – the latest in a string of perceived polling failures in recent history.
For two years, polls put the left-leaning Labor party ahead of the more conservative Coalition. While the gap progressively shrank in the six months ahead of the election, the figures were consistently in favour of a Labor win. Of the 16 polls held since the election was called last month, all went in favour of the Labor party when responders were asked to pick from the leading two parties.
On the eve of the election, four key opinion polls had Bill Shorten’s Labor at a 51-49 over Morrison’s Coalition. Exit polls and betting sites also got it wrong, with Sportsbet set to pay out more at least A$5.21 million.
As the picture of a Coalition victory began to emerge on Saturday evening, ABC’s election analyst Antony Green called it “a bit of a spectacular failure of opinion polling”. Other politicians and commentators have since joined in the criticism.
Kevin Bonham, an independent analyst from Tasmania says this was worse than the failure to predict Donald Trump’s US election win in 2016. There, pollsters accurately predicted Hillary Clinton winning the popular vote, but failed to properly poll key local areas that were assumed to be safe Democrat and instead swung to Trump.
A single poll can be wrong by two or three per cent because of random sample error, but Bonham says it is suspicious that all the polling firms were repeatedly getting similar, incorrect results. “It’s vastly improbable that would happen by random chance,” he says.
Instead, he suggests that polling companies might have engaged in statistical smoothing, or herding, of the figures because they want to avoid being embarrassed by having a different result to others. He cited one example in Australia where a polling group broke with its competitors and called the wrong election result, which caused them to lose their contract.
Nevertheless, Bonham says the idea that polls are getting worse worldwide isn’t borne out by the research. Australia has typically had “very accurate” national polling, he says. That may be because voting is compulsory – if pollsters don’t have to try to anticipate who’s going to vote, they can just focus on how they are voting.
This isn’t straightforward though. Shaun Ratcliffe at the University of Sydney says one of the difficulties with accurate polling is that it is increasingly difficult to get people to respond to opinion polls. Fewer houses have landlines, especially among certain demographics such as young people, and mobile phones make it easier for people to screen calls, he says.
Ratcliffe also believes that an overlooked problem is accurately polling people who speak a language other than English at home, which accounts for about one in six Australian citizens.
Despite this, he says opinion polls are getting more accurate in Australia. “It’s a bit like an arms race”, he says – ss demographics change and technology shifts, pollsters must update their methodologies.
“I think it’s easy to overact to one bad result, but we have to remember that there’s a common factor with all of the high-profile misses – they were all very close elections,” says Ratcliffe, pointing to the UK 52 to 48 per cent result in favour of Brexit, which also came as a polling shock in 2016 . It seems that whenever there is a highly-polarised vote, pollsters will have to cross their fingers.