Polls and Bias

There’s not much talk is about poll bias. But, there’s much fun in talking about polling inaccuracies, and to a lesser extent the inner workings of polls. But, nevertheless, let’s look at polls through the same prism of economic incentives that we’ve previously viewed the media.

1) unlike the media, polls are paid to be accurate

Many people forget that polls do not exist for advertising dollars, but rather exist because someone will pay them for the data they create. Now it may be that some polls (the larger ones) are paid simply to create headlines, for which the big media will pay them. So large swings in their polling results may be economically useful. This is why Zogby and Gallup are so wild. It is also why these two pollsters are so often on television explaining their polls – they get paid for that too.
But most polls are paid for by politicians seeking their data. When people complain about not being privy to the “internals” of a poll, what they really mean is that they weren’t willing to pay for the internals (media outlets pay big bucks because once they publish the value is lost). Politicians need accuracy and they pay big money for it. And they need the data keyed to particular demographic and geographic internals, and they pay bigger money for that.

2) polling is an artform

Well, polling is at least part science and part art. The art is particularly in the “likely voter” model, while the science is in the distribution of data from the model created. The art comes in predicting which people will vote in a given election. Some, like the Gallup traditional (one of its three models), predict based solely on previous voting patterns. This has obvious pros and cons. Others rely more heavily on registration patterns for new voters and other “excitement” polling to generate their models. This is more typical.

3) polling is erratic, so in come the averagers

The fact that polls may be erratic does not detract from their value to the campaigns. Because often an outlier poll can pinpoint its own model flaw, and retain the value of the internals. For example, one recent TPP poll showed a very close race. It subsequently, however, released data that showed a 78% inclination for McCain in the age 18-24 demographic. This violated common sense, and thus renders the overall poll numbers useless. But the data for other demographics remained viable and thus retained its monetary value.
This is where sites like RCP, Election Projection, and 538 become useful. They gather polling data and then attempt to aggregate to reach consensus. They are more susceptible to bias however, as they must weight each poll in relation to the others (or don’t weight and count on outliers to cancel each other out, which is what RCP does). Nevertheless, these sites can provide a better overall picture of races and provide valuable commentary on each poll’s usefulness. 538 is the site which pointed me to the TPP problems noted above, for instance.

4) polling may be generally tilted democrat

I’m still working on this, but in general, I think that because polls must predict turnout, the voting tendencies of groups must be the primary source of modeling. This means that if some groups are difficult to predict, more error will obviously occur in predicting that group’s behavior. It is suggested that republicans are a more predictable voter. Therefore, sites that use “excitability” polling and “voter intention,” may find that they consistently over-poll democrats. But tryin to model to avoid this would mean subtracting votes subsequent to the objective model based on party affiliation, which would a) look bad, b) be impossible to verify, since it assumes that the subject voter is lying/mistaken.

Why is this part of the American Romance? Similar to the media, polls are in the business of giving the people what they want and getting paid for it. They provide a valuable service to politicians, while giving the average American something to talk about. That’s American romantic, baby.