Getting Facts From Statistical Mushiness

In the course of my continuing research into the state of our current challenges, I came across a blog entitled, "The True Conservative." Through some means of logic, self-invented I think, the author of this blog, Steve Roth, has decided that the "True Conservatives" are actually Democrats.

While this in an interesting concept, and one I'll probably deal with in a later post, I wanted to spend a little time on how he uses statistics (polling data) to come to the conclusion that even Republicans (he calls them Pups) don't like Republican policies. His conclusion:

Example: Only 38% of Republicans support Republican tax policies.

Tell them it's the "Republican" policy position? All of a sudden 66% support it. Whodathunkit...Their problem is that even Republicans don't like Republican policies. They prefer Democrats' policies—except when they're told that the policies they approve of are Democratic policies.

Now that's interesting, isn't it? I thought I'd take a look at the source, and see if I came up with the same conclusion.

First of all, the sponsor of this survey is NPR, a more liberal outfit than say, Fox News. So I'm a little suspicious of the conclusions, but giving them the benefit of the doubt, I thought I'd take a deeper look.

What I'm looking at are two charts, but Mr. Roth comes to the wrong conclusion. The study premise is that two different groups were asked the questions, one identifying the question as a partisan position, the other non-partison. They did not ask the non-partison question, then identify it as partison to the same group.

In addition to this wrong conclusion reached by the liberal Mr. Roth, you must read and analyze the questions on page 31 of the study. Then study the charts on page 32. The questions are so close to each other, I'd have a hard time picking the best one. And the design of the survey is based on this:

On four issues - the economy, Iraq, foreign trade, and taxes, we read voters the issue positions of both political parties and asked them which statement came closer to their own views. Half of the respondents were read a version that included the Republican and Democratic labels, while the other half received questions without partisan cues.

To back up a minute, the first part of the survey, based on a telephone interview of 800 likely voters, has a margin of error of 3.46%, 95% of the time. That is a large margin of error, and many of the charts are statistically insignificant.

So in the second part of the study, the survey sample has actually dropped to 400, instead of 800, because the reseachers divided the groups in half. This increases the margin of error. In my experience, 400 likely voters is not a large enough sample.

So based on just this simple analysis, the study is inconclusive. Mr. Roth, what you are trying to tell us just may not be true, so don't write about it as if it is a fact. At this point, any honest researcher would conclude that more study is needed.

(Just for the record, I have a background in doing research of this type, and have taken statistics at the graduate-school level).