The Time Warp & Survey Design

Let’s do the time warp again or so they sang in The Rocky Horror Picture Show:

How does this relate to survey questionnaire design? Time is one asset we can never get back so those of us involved in data collection must be conscious not only of the time it takes a respondent to complete our survey, but also the types of items we employ in our questionnaire design.

It is common in customer satisfaction programs to measure overall satisfaction and the satisfaction with individual elements of the purchase or service exchange. The methodology for this technique has been almost standardized at this point, but one aspect under the researcher’s control is the time frame of reference. I am referring to the window of time you ask the respondent to consider. For example, you may ask:

In the last three months have you purchased a McJoyous Meal?

The follow up to this should the respondent answer in the affirmative would be a series of questions asking about the level of satisfaction with said McJoyous Meal. Again this may seem straightforward; however, it becomes a more difficult challenge for the respondent to accurately convey their sentiment for an event that could have happened any time within the prior three months.

The respondent’s ability to recall diminishes with time. It also is a function of the nature of the purchase with significant purchases (e.g. car, home, vacation, etc.) having greater likelihood of recall than routine transactions such as changing an address on your credit card company’s website. The visual below shows an example from a recent study I completed. The rows in this matrix were masked based upon a response to a previous question about which contacts I had with my credit card company in the prior three months.

Time

As you can see, if the contact was last week then my recall would be far better than if it were two months ago. Compounding the issue is the differences in time frame for the two contacts, which could easily lead to skewed data.

The best practice is to conduct satisfaction measurements in waves with the intervals being a function of the particulars of your product or service. Transactional measurements should be conducted why you have someone on the phone or the website. Significant purchases, a new car for example, can go longer, unless your focus is on the sales experience itself.

Consider the time frame you are asking your respondents to work with before designing your satisfaction measurement program. Avoid having to do the time warp again!

blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter