My career in medicine and science has taken me down some interesting paths, and one of the paths it took me down led to a college classroom where I looked out over a sea of faces that expected to learn something from me – specifically, they expected to learn the basics of the science of Statistics.
The fall of 2006 was just such a time, and like everything else in my life, the timing was perfect. As the election season was heating up, we were discussing polling. I told my students, a class of about 60, that I didn’t believe the polls were accurate that year, because they weren’t weighted for cell phone bias. To drive the point home, I asked my students a series of questions.
First I asked how many of them had a cell phone. Every hand went up.
Next I asked them how many also had a land-line phone. About 3/4 of the hands went down.
“Okay, put your hand down if that land line has a computer plugged in but no telephone.” This left only four hands in the air.
“And how many of you have been called by a pollster?” Not one hand stayed up.
“And how many of you are going to vote?”
All the hands went back up.
“No one is polling you guys,” I explained. “But you’re all going to vote. I think we are in for a game-changer of a year, and I think pollsters are going to spend the next two years scrambling to weight their polls for cell phone bias.” Then we talked about the science of polling. “Never take a polls numbers at face value. You have to look at the internals and the methodology. If a poll is only calling people with landlines, they are not producing an accurate poll. They are leaving out wide swaths of the population when they only poll landline phones. They leave out students and other relatively mobile demographic groups. They are calling up older, whiter, homeowners, and they are getting results that don’t reflect the population at large.”
I was right. The pollsters did scramble to fix that flaw before 2008, and they managed to hold their own, but the cell phone bias was still evident after the dust settled. And that bias is turning out to be quite a mountain to climb.
In a new analysis sure to add to the uncertainty about the upcoming election, the Pew Research Center reports that a large number of pre-election polls might be biased.
Polls that don’t interview people on cellphones are producing potentially inaccurate results, according to the Pew study. The vast majority of political polls today only interview on conventional, landline telephones.
Looking at their most recently released poll, Pew shows that a 7-point Republican advantage on the generic congressional vote question would have been a wider 12-point lead had they not included cellphone interviews. Three of four other Pew polls this year would have shown similar tilts toward the GOP, leading to Pew’s conclusion that “the bias [from not using cellphones] is as large, and potentially even larger, than it was in 2008.”
Nationally, Pew and some others (including The Washington Post) interview on cellphones, but few state- and district-level polls do so. Almost no automated polls include cellphone samples, in part because of the legal prohibition against having computers dial cellphone numbers. Approximately 25 percent of all U.S. adults are “cell only.”
Pew has been tenacious about researching cell phone bias in political polling, and while their latest study is not a broad, stinging rebuke to the art and science of polling, it does raise serious doubts about the accuracy of the data that is fueling speculation about a republican tsunami next month. The farther down the chain the polls take place, the more likely they are to be polling a disproportionate number of older, whiter, more conservative voters and skewing the polls toward predicting an inevitable takeover of the House by the republicans.
But I don’t buy it. I don’t think their polls are accurate. But there is a catch – they don’t have to be accurate if they can depress Democratic turnout. And it becomes a self-fulfilling prophecy if you say “what’s the point?” and stay home on November second.
Crossposted from They Gave Us a Republic