When designing survey quesitons, it's always important that you clearly define exactly what it is you are asking, especially when there is context given in the question itself. Take the following sample survey question found in many demographic surveys:
This would be a perfectly acceptable question to ask in conversation, but for a written survey question, the problem lies in the fact that the question is asked first, and the definition is asked second. Research shows that in self-administered surveys, respondents read the question, believe they've understood what is being asked, skip the context, and then answer the question. Each respondent may have their own definition as to what jobs fall under the "Legal Professionals" umbrella, and if they are typically skipping the context of your question, you are left with inaccurate and unreliable survey data.
Fortunately, the fix is an easy one: simply switch the order of information you are presenting, putting the context first, then ask the question. Now that all of your survey respondents understand the context of the question, your data will be much more reliable, thus making your survey report all the more accurate.

This would be a perfectly acceptable question to ask in conversation, but for a written survey question, the problem lies in the fact that the question is asked first, and the definition is asked second. Research shows that in self-administered surveys, respondents read the question, believe they've understood what is being asked, skip the context, and then answer the question. Each respondent may have their own definition as to what jobs fall under the "Legal Professionals" umbrella, and if they are typically skipping the context of your question, you are left with inaccurate and unreliable survey data.
Fortunately, the fix is an easy one: simply switch the order of information you are presenting, putting the context first, then ask the question. Now that all of your survey respondents understand the context of the question, your data will be much more reliable, thus making your survey report all the more accurate.


Comments for Survey Design Pitfalls: Question Context First