A report by three high-profile academics with expertise in public opinion research who were commissioned to look into polling failures that marred the 2017 Calgary municipal election campaign sharply criticizes the role of the city’s media in covering and sponsoring the polls.
Christopher Adams of the University of Manitoba, Paul Adams of Carleton University, and David Zussman of the University of Victoria singled out Postmedia, whose Calgary Herald and Sun newspapers sponsored a controversial series of polls that wrongly indicated a conservative candidate was dramatically in the lead, for particular criticism.
Professors Adams, Adams and Zussman were also critical of the polls that contributed to the controversy, in particular three demon-dialler surveys done by Mainstreet Research of Toronto for Postmedia that put Conservative candidate Bill Smith far ahead of incumbent Mayor Naheed Nenshi in the October 2017 campaign, in one instance by 17 per cent.
“The panel heard that these polls, which received the greatest media attention during the campaign because of their number, their startling results and their association with the two Calgary dailies, significantly affected he course of the campaign,” the report says. “They threw Nenshi’s campaign on the defensive, gave impetus to Smith’s campaign, and possibly doomed the prospects of another candidate, Andre Chabot, who Mainstreet’s poll suggested was not a close contender.”
Nevertheless, on election day on Oct. 17, as is well known, Mayor Nenshi won by a margin of 7.65 per cent. In the wake of his election victory, some of the mayor’s supporters accused Postmedia of being part of an unsuccessful campaign to roust Nenshi from office through biased reporting that assigned undue credibility to flawed polls.
As the report put it, “the Mainstreet polls … triggered an acrid debate in the media and on social media, in which the Nenshi campaign attacked the firm’s motives and independent academics questioned its results and methodology.”
Meanwhile, all through the campaign, “Mainstreet executives responded with unshakable confidence in their results and attacked their critics, often in personal terms, at one point suggesting there would be ‘payback’ after the election results were known,” the report says.
The trio of professors released their 70-page report independently early this morning. It had been commissioned by the Market Research and Intelligence Association, a national polling-standards group. It was never published by MRIA, however, which unexpectedly announced last week it was in financial trouble and was almost immediately going out of business. MRIA ceased operation on Tuesday, July 31.
On Postmedia’s coverage, the independent panel’s members said, a review of media coverage showed “Postmedia in particular was not critical enough in its reporting of polls for which it was partly responsible.
“Moreover, Postmedia did not share with its own readers concerns it had about the polls and the degree to which Mainstreet was altering its methodology to address them,” they went on.
They found that while Postmedia and other media operations reported on the dramatic discrepancies between the Mainstreet polls and others published during the campaign, “that coverage was not technically sophisticated and would not have left readers fully equipped to evaluate the polls.”
The panel noted that two other polls, one done for a group that supported a light rail transit line and the other for a long-term Canadian academic project, were off base too, but not as dramatically as Mainstreet’s, and in the right direction in that they both showed Mayor Nenshi in the lead.
“The Mainstreet polls, powered by the dominance of the two city dailies, and the ricochet effect it had through its reporting in other media outlets and social media, created a dominant narrative for the campaign: that the election, in short order, had become a two-horse race between Nenshi and Smith, with the mayor struggling to stay competitive.”
The panel also said it “found that Mainstreet’s overconfidence and its contentious style of public debate significantly contributed to the embarrassment of the industry when the results were proven to be radically mistaken.” The authors added that the Toronto polling firm’s public confidence in its results during the campaign “contrasted with internal concern about its results that led to adjustments in methodology.”
Those technical changes, the authors said, made some problems worse — in particular Mainstreet’s difficulties contacting a representative sample of younger voters. ‘When the election proved to have an unexpectedly high turnout, especially among young people, these vulnerabilities were yet further exposed.”
The three professors made 10 recommendations, many of them tied to actions that should be taken by pollsters, Mainstreet in particular, in conjunction with MRIA — which is going to be a problem now, seeing as MRIA has ceased operations.
Recommendations that could still be implemented, even without a Canadian polling standards organization, included:
- For media to fully disclose their commercial and financial relationships with pollsters when obtaining polling data on the basis of exclusivity
- Application of “normal journalistic context and scepticism” when reporting on polls obtained through exclusive relationships
- Use of a standardized disclosure checklist that all polling companies would publish with their polls, including an accountability tool for use by the public
With particular regard to Mainstreet, the panel had recommended the company work with MRIA to choose “a neutral academic auditor” who could evaluate their practices and adherence to standards with the goal of welcoming them back into respectable polling circles. That will be harder now without any Canadian polling standards body.
A recent poll by Mainstreet of Alberta voters’ intentions said the United Conservative Party led by Jason Kenney is dramatically ahead of Premier Rachel Notley’s governing New Democratic Party, leading to much crowing on social media by UCP supporters.
Click here to read the full report by Adams, Adams, and Zussman.
This post also appears on David Climenhaga’s blog, AlbertaPolitics.ca.
Photo: David J. Climenhaga
Like this article? rabble is reader-supported journalism. Chip in to keep stories like these coming.
Support rabble today!
We’re so glad you stopped by! Thanks for consuming rabble content this year.
rabble.ca is 100% reader and donor funded, so as an avid reader of our content, we hope you will consider gifting rabble with a donation during our summer fundraiser today.
Whether it be a one-time donation or a small monthly contribution, your support is critical to keep rabble writers producing the work you’ve come to rely on as a part of a healthy media diet.
Become a rabble rouser — donate to rabble.ca today.
Nick Seebruch, editor
