Differences in Question Layout = Differences in Survey Responses

Every question we put into our surveys is a delicate instrument created to gauge a behavior or opinion. The way we choose to form our survey items can lead to detectable differences in the types and quality of responses we receive. In particular, the layout of our questions can go a long way towards getting high quality data, or the opposite.

Even though we may not realize it, every question in a survey contains both verbal and nonverbal cues that may influence our participants behavior. Nonverbal clues we can insert into our surveys can be graphical, numerical, or symbolic languages that convey meaning to our respondents. For example, using a dollar sign ($) to indicate we'd like a person to respond in US dollars. Verbal language clues include the phrasing we use to write our question items. 

Some studies have revealed that the more bells and whistles we put in our surveys (e.g. many different colors, symbols, pictures, and videos) affects what respondents read, the order in which they read it, and, ultimately, their comprehension of the information we are asking them to provide to us.

Additionally, changing number values attached to question item scales can change the answers of our participants. For example, using a scale of 0-10 has been found to lead to lower scores than using a 1-5 format.
blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter