Asking for the Truth

In survey research, be it consumer marketing research, B2B or that for non-profits, we rely on the assumption that participants, more often than not, will respond truthfully. We know, from years of social psychology research that questions regarding sensitive topics will be responded to differently if the survey is administered in person versus online, by mail, or over the phone. Sensitive topics include finances, sexual behavior, alcohol or substance use, etc. The presence of an interviewer will prompt certain individuals to downplay perceived negative behaviors (e.g. number of alcoholic drinks consumed) and play up positive behaviors (e.g. thoughts on charitable donations).

In the case of self-reported online surveys is it wise to prompt respondents to answer truthfully, to fully read the questions, and in short give it their best shot? My answer is yes. Individuals do not like being told what or how to do something, but we can generally handle a simple reminder. The graphic below is an example of an upfront statement provide to panelists participating in a recent survey. It is designed to inform them of the penalty for not providing accurate responses. It does so using a ‘positive sandwich’ approach of a positive statement, followed by a negative (in this case the penalty of losing an incentive), finishing up with a declarative statement.

truth

Breaking it down, the survey author stated they value the respondent’s opinions and their time. This is a critical statement as it sets a positive tone. The author follows with a request for the respondent to read and respond carefully. This is a pitch for data quality. The data from online survey platforms should pass through the same quality checks afforded data from other sources. These quality checks include reviewing missing data, looking for signs of straightlining (answering matrix questions by choosing one number for all rows) or speeding (taking far less time than the norm for survey completion.

The final part of this opening is a call for conscious participation and an affirmation in the form of a Disagree/Agree response. I am sure those that answer ‘disagree’ will be screened out. My final thoughts are; if you are surveying the general population then this type of focusing technique is appropriate. If you are surveying members of a religious group, for example, and can reasonably expect truthful responses then it may not be necessary. Then again a simple reminder seldom hurts.

blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter