Sometimes Opinion Polling Creates More Questions Than Answers

Today, the New York Times Week in Review published an article on public opinion polling and question wording. The article discusses how question wording can affect polling stats, making it appear as if Americans are waffling on certain issues, or cant make up their minds. When in reality, the culprit of conflicting poll results may truly be question wording.

For example, in four recent polls on the issue of the current health care plan proposal conducted by four separate research companies explained the plan to respondents in three different ways. Company A explained it as a government administered health insurance plan — something like the Medicare coverage that people 65 and older get, Company B described it as a government-sponsored public health insurance option. Meanwhile, Company C touted the health care plan as a government health insurance plan, and finally the fourth poll used the description of a government-run health insurance plan.

What was the result? The four different question wordings resulted in four different sets of results, with results that said anywhere between 44 and 66 percent of the population supports a health care proposal like the one described in their particular poll.

Well, any seasoned researcher and business person knows that a range of that magnitude isn't telling you much. If business decisions are on the line, its important to have results you can be confident about.

So the next time you are reviewing the results of your poll or survey, also review the question wording used in your instrument. The opinions of your clients may not be as fickle as it appears – question wording could be the key to the variance.
blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter