How much faith do you put in the results from news text polls? According to a Fox News poll 98% of their viewers legitimately believe polling results, and 98% of MSNBC viewers believe the text polls favor public opinion. If you believe these polls give valid, representative survey results then you have not been paying attention to all my survey best practices tips. I think Jon Stewart does a much better job explaining how flawed this thinking is, so I'll let him explain:
Sure, a cable news text poll is good for marketing and PR spin, it certainly bolsters your argument that we need to "increase taxes" if you're in favor of increasing taxes on the top 1% of Americans. Or conversely, it makes your point that the majority of Americans are against increasing their taxes. This is no surprise. However, it's pretty worthless beyond that.
If for a moment you want to put aside the problems with their survey sample not being representative, these questions are designed to get a specific answer. Do we need a tax hike? There are very few people who like and enjoy paying taxes, so who do you think is going to say use to this Fox News poll? I'm guessing not very many - but oh wait, I know how many 8% said they thought we needed a tax hike. If you watched the Daily Show clip, you know MSNBC did a similar poll asking if it's fair to tax the top 1.2% of Americans to pay for health care reform. This will blow your mind: 94% agreed it was fair. So what's the true feeling? Probably something along the lines of, I don't want to pay more taxes, but it's okay if you make other people pay more taxes. And the 6% who said it wasn't fair were probably in the group who would begin paying more taxes. I'm sure you see where this is going, asking leading questions leads to unreliable data. It shouldn't be used to make business decisions.
It's also important to ask the question: How representative is the survey sample? If you think for a minute about what the chosen survey method of data collection, you're probably half way there. In all these cable news polls they were using non-probability sampling methods, which isn't bad if they're carefully planned. However, when you couple that with the fact that different types of people prefer different cable news networks, and even different shows, now you've really raised question with the validity of the data. You may be able to claim the poll results are an accurate representation of how the viewers of that show feel (but I would raise question with that as well), but that's as far as it goes.
Thanks Jon Stewart for helping us see the errors of our way in believing these news polls. Hopefully, the 98% of you who believe them know now to question such polls - particularly when there's so little (barely any!) variance in the data.
Sure, a cable news text poll is good for marketing and PR spin, it certainly bolsters your argument that we need to "increase taxes" if you're in favor of increasing taxes on the top 1% of Americans. Or conversely, it makes your point that the majority of Americans are against increasing their taxes. This is no surprise. However, it's pretty worthless beyond that.
If for a moment you want to put aside the problems with their survey sample not being representative, these questions are designed to get a specific answer. Do we need a tax hike? There are very few people who like and enjoy paying taxes, so who do you think is going to say use to this Fox News poll? I'm guessing not very many - but oh wait, I know how many 8% said they thought we needed a tax hike. If you watched the Daily Show clip, you know MSNBC did a similar poll asking if it's fair to tax the top 1.2% of Americans to pay for health care reform. This will blow your mind: 94% agreed it was fair. So what's the true feeling? Probably something along the lines of, I don't want to pay more taxes, but it's okay if you make other people pay more taxes. And the 6% who said it wasn't fair were probably in the group who would begin paying more taxes. I'm sure you see where this is going, asking leading questions leads to unreliable data. It shouldn't be used to make business decisions.
It's also important to ask the question: How representative is the survey sample? If you think for a minute about what the chosen survey method of data collection, you're probably half way there. In all these cable news polls they were using non-probability sampling methods, which isn't bad if they're carefully planned. However, when you couple that with the fact that different types of people prefer different cable news networks, and even different shows, now you've really raised question with the validity of the data. You may be able to claim the poll results are an accurate representation of how the viewers of that show feel (but I would raise question with that as well), but that's as far as it goes.
Thanks Jon Stewart for helping us see the errors of our way in believing these news polls. Hopefully, the 98% of you who believe them know now to question such polls - particularly when there's so little (barely any!) variance in the data.
Powered by Compendium Blogware
Comments for Why You Shouldn't Believe Cable News Poll's Results