Survey Design Pitfalls: Leading Questions and Loaded Words

You should take great care to avoid using leading questions and loaded words when you design survey questions.  These two faulty sample survey designs can do more harm than any other possible hazards during the survey creation process.  They can bias respondents’ views (and hence their answers) to the point of making your data unusable.

Leading questions bias respondents by subtly directing them toward particular answers.  These items usually provide judgments about ideas or concepts before the actual questions are asked -- or the wording of the questions may just be biased from the outset.  Leading questions can make respondents feel as if there is an obviously "correct" response, and that they would be foolish to answer otherwise.

Loaded words are those which carry overtones or connotations that predispose a survey respondent to think in a certain way.  If researchers use overly strong words (positive or negative), or use labels with clearly judgmental implications, this can bias respondents by priming them for certain mindsets.  Consider the following example of a leading question filled with loaded words:

Most people feel that $8.00 is way too much money to pay for a simple burrito. 
Would you pay $8.00 for a burrito?

We can easily tell that the purpose of this "question" is to bias respondents into saying they would not pay eight bucks for a burrito.  There is a clearly derogatory sentiment contained in the question itself (leading).  And with the inclusion of phrases such as most people feel, way too much, and simple burrito, respondents would have to be irrational fools to admit they might pony-up the $8.00 (loaded).

This of course is an extreme example, but if you’ve been selected for a political poll, you know that there are agenda-based questions out there that are almost as egregious.  When writing survey questions remember: your results are usually only as good as your questions!  By avoiding leading questions and loaded words, you can be more certain that your questionnaire results will accurately represent the views of you customers, employees or public opinion survey samples.
blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter