Common Survey Pitfalls: Leading Survey Questions

We've talked about two common issues that crop up when writing survey questions: complex and ambiguous questions. Sometimes, they can be hard to spot. Thankfully, our last three question pitfalls are easy to identify and correct.

Leading questions
This is a common question that can pop up when writing satisfaction questionnaires or market research surveys. When we're writing employee questionnaires, product feedback surveys or other market research instruments, we want to make our concept, product or organization look good. This can cause use to write questions that lead the respondent to a particular answer. I don't need to tell you this means you're collecting unreliable survey data. People like to agree and fit in, so if you lay out a correct answer for them, it's unlikely they'll pick the wrong one. Survey questions shouldn't work like that. There should never be a right and wrong answer.

Understanding you don't want to include leading questions in your next feedback form is easy enough. So let's look at a few example questions of what not to do.

Sample Public Opinion Poll Survey Question: Do you personally approve of having your tax dollars spent to support illegal aliens?
Sample Public Opinion Poll Survey Question (Leading)

Sample Public Opinion Poll Survey Question: Do you personally approve of having your tax dollars spent to support illegal aliens?
Sample Public Opinion Poll Survey Question (Solution)

Sample Consumer Survey Question: Experts believe that all consumers should comparison shop. Do you agree?
Sample Consumer Survey Question (Leading)

Sample Consumer Survey Question: All consumers should comparison shop.
Sample Consumer Survey Question (Solution)

Sample Consumer Survey Question: Not all consumers should comparison shop.
Sample Consumer Survey Question (Solution)

To transfer a leading question into an unbiased question you just need to remove the leading words or phrases. These are the ones that make a judgment or imply stupidity if you disagree.

In the first leading survey example, we see that there's a budget problem and our tax dollars are being wasted. So of course we will disapprove of more waste. If you remove the commentary before the question, it becomes more neutral.

In the second leading survey question example, there is the implication you're dumb if you disagree because experts say this is what you should do. By removing the statement about experts, you create a neutral question. It's also an online survey best practice to reverse the wording for half your survey sample. This helps to remove respondent biases (such as selecting the first or last response option).

Tomorrow we'll look at loaded questions, which are similar to leading questions.
blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter