People often worry about the integrity of their survey data and how to protect it. While there's no magic solution, here are a few tips to help you identify those respondents who are not thoughtfully answering your survey questionnaire. Chances are you spent a lot of time creating a questionnaire to meet your survey project goals, ensuring the result is quality responses should be at the top of your list. Here are a few things you can do during the survey creation to ensure the integrity of your survey results:
1. Question respondents' qualifications. Some people refer to this as a knowledge trap. The idea is to verify the respondent is the type of professional they claim to be (and you need for your survey). Typically these questions belong towards the top with other qualifying questions. For example, if you're conducting a market research survey targeting educators you may ask a question specific to their field. But you can also use this tactic for product evaluation surveys or customer questionnaires by asking questions related to your product or service that only a client will be able to answer. Here's a sample question for a Cvent user:
2. Test respondents' logic. These types of questions are intended to catch Christmas Tree-ers and straight liners (those who give the same response to every question to speed things along). For this type of trap, you ask the same question multiple ways. Here are a example survey questions utilizing a logic trap:
Depending on how you use logic traps, it may be necessary to space them out. You wouldn't want to have my sample survey questions appear one right after another. It will irritate respondents because you're asking them the same thing twice and wasting their time. Don't be overly obvious, like I was, about it. When you're looking at your results, if someone said it was very likely they would buy the product in the first question but said it was very unlikely they would buy the product in the second, you probably have a problem.
3. Bring respondent attention back. Sometimes when you're completing a survey or questionnaire online you begin to go on auto-pilot only reading the part of the question or just skipping to the responses - particularly if the survey is long. A way to combat this tendency is to add some attention traps to your questionnaire design. An easy way to do this is to throw an unrelated attribute into a ranking scale. It forces the respondent to stop think for a second about what the question is asking and refocus. Here's an example:
Instead of throwing out a respondent because they fail one of these tests, I would suggest simply throwing out that specific answer. In other words, purge data at the question level not the respondent level. If they fail every test and their completion time is way off, perhaps their response is impacting the integrity of your results. After all, the goal of every survey, questionnaire or feedback form should be to answer a question and use the survey report to make decisions.
1. Question respondents' qualifications. Some people refer to this as a knowledge trap. The idea is to verify the respondent is the type of professional they claim to be (and you need for your survey). Typically these questions belong towards the top with other qualifying questions. For example, if you're conducting a market research survey targeting educators you may ask a question specific to their field. But you can also use this tactic for product evaluation surveys or customer questionnaires by asking questions related to your product or service that only a client will be able to answer. Here's a sample question for a Cvent user:
2. Test respondents' logic. These types of questions are intended to catch Christmas Tree-ers and straight liners (those who give the same response to every question to speed things along). For this type of trap, you ask the same question multiple ways. Here are a example survey questions utilizing a logic trap:
Depending on how you use logic traps, it may be necessary to space them out. You wouldn't want to have my sample survey questions appear one right after another. It will irritate respondents because you're asking them the same thing twice and wasting their time. Don't be overly obvious, like I was, about it. When you're looking at your results, if someone said it was very likely they would buy the product in the first question but said it was very unlikely they would buy the product in the second, you probably have a problem.
3. Bring respondent attention back. Sometimes when you're completing a survey or questionnaire online you begin to go on auto-pilot only reading the part of the question or just skipping to the responses - particularly if the survey is long. A way to combat this tendency is to add some attention traps to your questionnaire design. An easy way to do this is to throw an unrelated attribute into a ranking scale. It forces the respondent to stop think for a second about what the question is asking and refocus. Here's an example:
Instead of throwing out a respondent because they fail one of these tests, I would suggest simply throwing out that specific answer. In other words, purge data at the question level not the respondent level. If they fail every test and their completion time is way off, perhaps their response is impacting the integrity of your results. After all, the goal of every survey, questionnaire or feedback form should be to answer a question and use the survey report to make decisions.
Powered by Compendium Blogware
Comments for 3 Traps To Catch Bogus Survey And Questionnaire Responses