Tags

There’s been a bit of discussion about transparency when it comes to polling by Nate Silver over at fivethirtyeight.com and Mark Blumenthal at Pollster.com.

The mechanics of executing a scientific random sample poll are relatively straightforward, the logistics are a nightmare. The people involved need to understand what they’re doing (duh) and what it takes to get valid results.

You don’t need a massive building and gigantic corporate infrastructure to pull off an accurate poll, though the available logistical support inherent in those entities can help a lot. What is absolutely essential is a thorough understanding of the science and the art of polling and a phone bank (or other contact system) with intelligent well trained interviewers. Mom and pop operations can produce good stuff, it’s just that it’s a really tough grind to do it with a relatively small number of people day in and day out.

Transparency when you publicly release your data is very important. The thing is though, if a candidate’s campaign spends a small fortune on a poll and releases the data to the public they’ve just given their opponent some of the benefits of the poll for free. The crosstabulations in a campaign poll are used for voter targeting. See the conflict?

But, when you publicly release the results there’s an obligation to release a minimum amount of information about your methodology so the public and others in the opinion research business can gauge the validity of the those publicly released results.

Why bother? Without methodological transparency the authority conveyed by a poll labeled as such, which may or may not have actually been executed with a valid methodology, can have significant influence on the media and the public at large.

From 1994:

AAPOR Finds Frank Luntz in Violation of Ethics Code

Wednesday, April 23, 1997 — The Executive Council of the American Association for Public Opinion Research (AAPOR) announced Wednesday that a 14 month investigation found pollster Frank Luntz violated the Association’s Code of Professional Ethics and Practices.

AAPOR found Luntz, who heads the Luntz Research Companies in Arlington,Virginia, repeatedly refused to make public essential facts about his research on public attitudes about the Republicans’ “Contract with America.” In particular, the AAPOR inquiry focused on Luntz’s reporting, prior to the November elections in 1994, that his research showed at least 60 percent of the public favored each of the elements in the GOP “Contract.” When later asked to provide some basic facts about this research, Luntz refused.

AAPOR holds that researchers must disclose, or make available for public disclosure, the wording of questions and other basic methodological details when poll findings are made public. This disclosure is important so that claims made on the basis of opinion research findings can be independently evaluated. Section III of the AAPOR Code states: “Good professional practice imposes the obligation upon all public opinion researchers to include, in any report of research results, or to make available when that report is released, certain essential information about how the research was conducted…”

Yep, that “Contract on America” has had a significant impact on our national consciousness. At the very least we should all be given the opportunity to judge if that influence is valid or not.