Wednesday, December 21, 2005

Read polls more carefully 

I am not a pollster -- my forecasting gets done with observable data, not surveys* -- but I tend to believe people can't read poll data any better than they read, say, GDP data. And I have to now address my friends at Kennedy vs. the Machine for slamming the SCSU survey. In short, they aren't reading it carefully enough and letting the STrib's spin represent what the poll actually says. Let me explain.

Their evidence is a new Survey USA poll that shows a job approval rating for Governor Pawlenty of 59% (35% disapprove). The 39% that KvM cites from the SCSU survey (here are the results slides) is for his re-election. What was not reported in the STrib was the survey's own question on Pawlenty's job performance and the fact that it was better than the results for "your own state representative" and for own senator.

Excellent 7.7%
Pretty Good 37.4%
Only Fair 29.9%
Poor19.6%

46.1% saying excellent or pretty good isn't too shabby. And you're comparing this to a Survey USA question that asks to approve or disapprove of Pawlenty's work. It is quite possible those two polls are measuring the same thing and the four-branch choice the SCSU survey gives sorts the answers of "approval" that would have been given to the Survey USA callers into some of the first three branches.

Andy thinks the Survey USA poll is a little off, too, citing 50% of pro-choice voters approving of Pawlenty (and 32% of self-described liberals). But I don't think that means very much either. As a personal example, if someone had called me and asked if I approved of the job my former representative Joe Opatz -- the DFL guy leaving the seat that has been the object of l'Affaire Ek -- I'd've said yes, I approve. Would I vote for him? I don't vote purely on party ID, but it's a pretty fair bet I voted against him a few times. (Sorry Joe!) Voting isn't just about one's job performance but also about the alternative candidates placed before you.

Likewise, using the "feelings thermometer" that the SCSU survey developed as a predictor is hazardous. I'm not particularly pleased with several GOP leaders right now, both nationally and locally, but my preferences don't lie on a single scale and I don't draw challengers from a jar at random. Or, as my old public finance prof used to put it, "a thing is neither good nor bad save the alternatives make it so." So too with politicians.

*--not that I can't do surveys: I just had a paper published on remittances that includes survey data. But I don't predict things using them.

Disclosure: One of the faculty directors of the student-based SCSU survey reads here. I have not run this by him. For all I know, he will disagree with it.

Categories: , ,

[Top]