09/19/2010

Bias in Polling: the Cell Phone Challenge

Print More

Pew’s latest report (May 2010) notes a couple of important polling issues that arise as many people go “cell only” and pollsters continue to use phone polls. It is more complex and costly to include cell phones in a poll sample. I’ve bolded what I thought would influence how I consider election eve and other election talk this year and from now on.

Nate Silver, uses his “kitchen sink” approach to making estimates and predictions, partly in response to bias such as this. And Mark Blumenthal at pollster.com highlights the real problem with looking at polls from the past and trying to base the reliability of future polls on them:

Calling by cell phone adds considerable expense and runs up against a federal law that bars pollsters from dialing a cell phone using any automated means. For live-interviewer polls, that means more time consuming hand dialing of cell phone numbers. For those using an automated method — like SurveyUSA, Rasmussen and PPP — the regulation is a total barrier. The automated (IVR) pollsters simply cannot interview respondents by cell phone.

Here’s what Pew says:

While the current average estimates of bias are still small, even relatively small biases can have important substantive implications. For example, our dual frame estimates of congressional vote intention for the November 2010 elections show Republican and Democratic candidates running even (44% each). However, estimates based on the landline sample show a six-point Republican lead (47%-41%). Given the demonstrated sensitivity of the final partisan distribution of seats in the U.S. House to the overall partisan split in the vote, it matters greatly which of these estimates is more accurate. If the landline estimate is correct, historical precedent suggests that the Republicans would recapture control of the House. If the dual frame estimate is correct, that is less likely to happen.

via Assessing the Cell Phone Challenge – Pew Research Center.