We, like you, get asked to participate in web based surveys a lot. Recently, a retailer sent us a customer service satisfaction survey, and we couldn't help but notice that the questions were going to yield bad data.
Here are a couple of question pitfalls we encountered, plus advice on how to avoid them:
• Compound questions. Inexperienced survey writers often make this mistake. When creating your survey, only ask one question at a time. For example, "Please rate your satisfaction with our products and services," is confusing. If someone says they are extremely satisfied, do they mean just product satisfaction, services satisfaction or overall satisfaction? There's no way of knowing.
Asking these two questions will shorten your survey, but do so at the cost of yielding useless data.
• Open-ended questions instead of closed questions. A common example of this pitfall is asking "When were you born?" This could yield a variety of results. Most people will answer with a date, but how will it be formatted? Will they type out January 19, 2009 or 1/19/09? If they're from countries outside the United States, they may type the date first followed by the month.
Incorrectly using open-ended questions will cause unnecessary challenges for data analysis as well as useless data (for example, if someone should type in a response like "After World War II.") Make sure you define the parameters for responses in situations such as these.
• Assumptions. In the previous example, the surveyor made the assumption that everyone was going to respond a specific way. Big mistake. Spell it out. Don't assume respondents will know what you were thinking when you asked how they felt about "soap." What kind of soap?
• Industry jargon. Just because you know what you mean when using an acronym doesn't mean all of your survey respondents do. Even if you've accurately targeted your survey invitations, you should always err on the side of caution. Not only will jargon frustrate the respondent, but it will make it impossible for them to answer accurately.
The overall lesson here? Be as clear as possible in your online surveys. Use everyday language, explain the formats for entering responses, and give background information where necessary. A few tweaks to your questionnaire will provide much better feedback for analysis and decision making.
Here are a couple of question pitfalls we encountered, plus advice on how to avoid them:
• Compound questions. Inexperienced survey writers often make this mistake. When creating your survey, only ask one question at a time. For example, "Please rate your satisfaction with our products and services," is confusing. If someone says they are extremely satisfied, do they mean just product satisfaction, services satisfaction or overall satisfaction? There's no way of knowing.
Asking these two questions will shorten your survey, but do so at the cost of yielding useless data.
• Open-ended questions instead of closed questions. A common example of this pitfall is asking "When were you born?" This could yield a variety of results. Most people will answer with a date, but how will it be formatted? Will they type out January 19, 2009 or 1/19/09? If they're from countries outside the United States, they may type the date first followed by the month.
Incorrectly using open-ended questions will cause unnecessary challenges for data analysis as well as useless data (for example, if someone should type in a response like "After World War II.") Make sure you define the parameters for responses in situations such as these.
• Assumptions. In the previous example, the surveyor made the assumption that everyone was going to respond a specific way. Big mistake. Spell it out. Don't assume respondents will know what you were thinking when you asked how they felt about "soap." What kind of soap?
• Industry jargon. Just because you know what you mean when using an acronym doesn't mean all of your survey respondents do. Even if you've accurately targeted your survey invitations, you should always err on the side of caution. Not only will jargon frustrate the respondent, but it will make it impossible for them to answer accurately.
The overall lesson here? Be as clear as possible in your online surveys. Use everyday language, explain the formats for entering responses, and give background information where necessary. A few tweaks to your questionnaire will provide much better feedback for analysis and decision making.
Powered by Compendium Blogware
Comments for Online Survey Question Pitfalls—And How to Avoid Them