Survey Software

Order and Flow in Online Surveys

Friday, October 16, 2009 by Tyson Gingery
Although question order effects are believed to be stronger for interviews than for online research surveys, it is still important to take a look at your market survey format and how the questions are ordered throughout your survey questionnaire.  By following the general guidelines listed below, you can reduce possible order effects and response bias from your customer survey samples.

Start broad, then get more specific.  Ask your customers or respondents general questions about your organization and the concepts you are interested in first, then get down to the more detailed questions you want them to answer (such as those about individual products, specific preferences and how to improve customer service).

Ideally, you’d like to find out if the order of your questions can induce biases before you send it to your entire survey population.  Use online survey software that allows you to randomize parts of your online survey, and compare the results with those from predefined orders.  This is an excellent way to pretest your survey questionnaire for possible order effects.

Devote some time and effort to making your survey “flow.”
  You want the overall survey instrument to be cohesive, not disjointed or seemingly all over the place.  A big part of allowing for a logical flow throughout a survey comes from writing smooth transitions.  Let your respondents know that you want them to change gears.

Ask demographic questions at the end of your survey.

If you are asking sensitive questions (about topics like income, personal behaviors, etc.) try to place these toward the latter parts of your survey as well.  That way you get at least partial completions in the event that respondents change their mind and exit your survey before full completion.

Comments for Order and Flow in Online Surveys

Monday, October 19, 2009 by Tony Whyde:
Very interesting post. I think many people do not realize how important flow is to creating a positive respondent experience. It is good to think about the survey as a conversation guide - even if you are not using interviewers it should still sound like a conversation. One way to test for flow is to simply read it aloud to yourself or someone else. Also, another way to help improve flow is to group questions by topic if possible with, as you stated, smooth transitions. Good transitions cue the respondent that it is time to think about something else which helps to reduce order effects. Again, great topic and you make some excellent suggestions.

Leave a comment





Captcha