Because You’re Doing it Wrong

Pollster via CQ has a story on the disparity between the results of two different polls, one by Mason-Dixon and the other by Ipsos. The polls took place in Florida and showed some significant variation: In one, Democrat Kendrick Meek was up 14 points on Republican Jeff Greene while on the other, Greene had 4 points on Meek. Their take? Florida’s wacky demography.

[1] With its transient population, high elderly concentrations and mix of races and urban-to-rural lifestyles, Florida is one of the most difficult states for pollsters, Brad Coker of Mason-Dixon Polling & Research tells the Tampa Tribune in a story that tries to untangle why different companies come up with such different poll results. How different?

In their recent polling on the Democratic Party’s primary to pick a Senate nominee, Mason-Dixon showed Rep. Kendrick Meek 14 points ahead of Jeff Greene, while Ipsos showed Greene up 4 points.

[3] An important difference between the polls was the construction of the sample, reports the Tribune’s William March. Ipsos polled registered voters. Mason-Dixon polled people who had previously voted in prmiaries and said they were likely to vote in this one.

The Mason-Dixon outcome is “almost certainly more useful for the primary,” Ipsos pollster Julia Clark told the newspaper, but said Ipsos’ clients wanted a registered-voter poll that could provide a clearer general election picture.

What? Paragraph three is the story. Paragraph one, generally the set-up for the entire point of an article, is nonsense. Pollsters have archived voterfiles and archived polls, they know how to apply weighting to their data to account for variation in the demographics they find, and they definitely know how to tell calling houses that their job isn’t done until their sample consists of a certain demographic breakdown. No one could seriously say that an eighteen-point difference in polls was part of some variation in the sampling that couldn’t be statistically controlled.

To me, this really feels like it should be a story about the laziness of Ipsos’ client. Why would you prefer registered to voters to likely voters? Both types of voters can be pulled from the same Voter Activation Network files and probably cost the same or nearly the same to acquire. I can’t say there would be anything conspiratorial going on on behalf of Ipsos’ client – it just seems lazy. And it definitely strikes me as a little odd that CQ, my favorite political journalism site, wouldn’t jump all over this obvious discrepancy. Brad Coker is wrong. Demographic differences in sampling is very unlikely to account for these differences because statisticians have spent centuries figuring out ways to control for those differences. Registered voters and likely voters, however, are very different animals.

In my opinion, when the sample populations are systematically different from top to bottom, the results are incomparable. They’re statistical apples and oranges.

Advertisements

Tags: , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: