This morning I received an email survey invitation asking me to take an internet survey for a political party regarding my views on current policies. There were so many things wrong with the survey, it was really disappointing. If you're going to spend the money to create an online survey, pay consultants and send it out to your entire subscriber base, shouldn't the survey results be helpful? Shouldn't the survey yield accurate data? My response would be yes. Otherwise, you wasted your own time and money, but you also wasted the respondent's time (and in this case, my time). Here are a few take a ways from the survey:
• DON'T ask leading questions. We've talked about how to avoid leading questions that result in bad data before. Survey questions shouldn't pressure respondents to respond in a specific way. The goal of any survey should be to gather honest answers. In the case of this survey, every question pointed towards responding "Yes." Questions were worded in such a way that even if you disagreed with the policy, saying "No" made you feel un-American.
• DON'T leave out important information. Some of the policy questions they asked, I had never heard of before. If I don’t know what the policy is, I can't answer the question. Make sure to provide enough information that respondents can respond in an honest, informed way. If the questions were not leading, this may have not been an issue. But it's a good tip. When you need to provide additional information, remember to define before asking.
• DON'T ask every question under the sun. Your respondents are busy. Respect their time by only asking questions that are relevant. This survey asked an average of 10 questions about each policy, then additional questions about the party's strengths and weaknesses. Long surveys see higher abandonment rates, so keep them short.
And Finally...
• DON'T waste my time. This one is related to number 3, but it's important so I want to reiterate. Everyone is busy. I don't have 30 minutes to take this survey, and chances are your respondents don't care enough to spend 30 minutes on your online survey either. When I couldn't take it anymore, I gave up and abandoned the web survey. The survey administrators would have been better off splitting their lists using a random sampling method and asking each group about different policies.
• DON'T ask leading questions. We've talked about how to avoid leading questions that result in bad data before. Survey questions shouldn't pressure respondents to respond in a specific way. The goal of any survey should be to gather honest answers. In the case of this survey, every question pointed towards responding "Yes." Questions were worded in such a way that even if you disagreed with the policy, saying "No" made you feel un-American.
• DON'T leave out important information. Some of the policy questions they asked, I had never heard of before. If I don’t know what the policy is, I can't answer the question. Make sure to provide enough information that respondents can respond in an honest, informed way. If the questions were not leading, this may have not been an issue. But it's a good tip. When you need to provide additional information, remember to define before asking.
• DON'T ask every question under the sun. Your respondents are busy. Respect their time by only asking questions that are relevant. This survey asked an average of 10 questions about each policy, then additional questions about the party's strengths and weaknesses. Long surveys see higher abandonment rates, so keep them short.
And Finally...
• DON'T waste my time. This one is related to number 3, but it's important so I want to reiterate. Everyone is busy. I don't have 30 minutes to take this survey, and chances are your respondents don't care enough to spend 30 minutes on your online survey either. When I couldn't take it anymore, I gave up and abandoned the web survey. The survey administrators would have been better off splitting their lists using a random sampling method and asking each group about different policies.


Comments for Biased Surveys = Bad Data