Selection of a Survey Sampling Frame Impacts Results

A recent national poll released by Gallup highlights the importance of knowing how to properly choose your sample frame for your research effort. Gallup says the demographics of tea party supporters are almost indistinguishable from the rest of the country's demographics.

Taken by itself that seems okay, however, a national poll conducted by CNN back in February 2010 found that the Tea Partiers were male, rural, upscale, and overwhelmingly conservative.

Why the different results? The answer lies in the way the questions were worded for the two polls. CNN's respondent demographics come from activists (those that have given money, attended a rally, or taken a step of some other sort in support of the Tea Party), while the Gallup respondent demographics are for supporters (those who consider [themself] a supporter of the tea party movement). This explains why Gallup's demographics more closely resemble those of the larger American population.

In the CNN poll, those that offer verbal support for the tea party total 35%, and in the Gallup poll 28%. Taking into account the differences in question wording these percentages are pretty close.

Closely examining the way we word our survey questions, and (when reviewing poll results) how others worded theirs, may help to illuminate what may at first seem like inconsistencies between results.
blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter