Decoding Open-Ended Questions

Note taking | Image by Nutdanai Apikhomboonwaroot

At what nexus does quantitative and qualitative research meet? The meeting point is the open-ended question. This question type produces unstructured data that can be detail rich or as sparse as “N/A”.

Open-ended questions are a good addition to your survey. Employ them in stand-alone fashion or subordinate to a preceding structured question. For example, try an unstructured question following a Net Promoter Score (NPS) question with open-ends for the detractors, passives and promoters. Each group would be asked a different question based upon their NPS response. This can be an informative addition to your customer satisfaction measurement by providing detailed thoughts on how to improve your offerings.

Stand-alone open-ended questions allow the survey author the opportunity to create a momentary pause in what can be an unending stream of categories and numbers. The thinking around where to insert unstructured questions varies. Some pundits suggest placing them at the beginning, others say place them only at the end of the survey. I say place them where they make the most sense. One caveat, though, I am seeing more surveys end with a question asking the respondent to share feelings about their survey experience. If you are open to the good, bad and the ugly then this can be a solid source of feedback to improve your game.

Ok, what’s the downside? Structured questions are easy to code and analyze. Unstructured questions are not. They require either a savvy text analytics tool or a diligent researcher with extra time on his or her hands to code the responses. Fortunately most online survey platforms are adding text analysis to their offerings. These tools can streamline the process of extracting meaning from unstructured data, but they are not fool proof.

If you are thinking old-school then you will be looking through the comments and extracting common themes (e.g. customer service is good or bad, price is high, etc.) There can be multiple themes in a respondent’s comments. These themes will then become either categories in an existing structured question or stand-alone questions in themselves. The goal is to be able to quantify the themes and tone (positive, neutral or negative) found in open comments. I find the use of a spreadsheet to be most helpful in sorting through the comments.

Read more about Cvent's Text Analytics

blog comments powered by Disqus
Crimes in Design Webinar
Subscribe to our Monthly Newsletter