Measuring Customer Satisfaction

Does your customer survey have "survivor bias"? I'm betting it does. Why? Because when most survey designers create business questionnaires or client satisfaction surveys we only collect feedback from individuals who are still customers. What about lost business? Why aren't those customers who left asked to complete your client feedback form? You better believe they have feedback.

In her post at the Dimensional Research blog, Customer Satisfaction Surveys: Avoid "Survivor Bias", Diane Hagglund defines "Survivor Bias" as drawing conclusions only from data that is available or convenient and thus systematically biasing your results. AKA biasing your survey sample by only asking "happy" customers. They may not be over the moon about your product, but they're at least happy enough with your offering if they're still paying you.

I agree with Diane on this one, it's pretty ridiculous to allow this bias to creep into your customer market research. It's easily avoided considering you should have all the customer data you need to send them the same client survey template. Make sure you're measuring client satisfaction among customers who left you for a competitor as well as those who simply decided not to renew the service (and didn't go with anyone else).

When you begin writing customer survey questions to create client questionnaires or update your annual template for a client satisfaction survey make sure you think about your former customers. If you don't know who they are, this is a good time to find out. Thank goodness for internet research software and customer insight solutions to make the act of surveying clients a little easier.

Yesterday, I shared the first 5 tips from the Meetings & Convention article, Survey Science: How to craft more effective attendee evaluations. No one can deny the importance of measuring customer satisfaction, so why wouldn't you want to conduct a post-event survey to measure attendee satisfaction? Particularly if it's an annual conference, meeting or tradeshow. If you want attendees to keep coming to your events, you need to make sure you're gathering feedback on what worked well, what didn't, what should be included next year.

In continuing with the fifteen survey best practices shared in the article, here are the next five:

6. Be careful with demographic survey questions. This tip comes from our very own Eric Eden, VP of Marketing here at Cvent. Survey respondents don't always like to share all their demographic information on surveys. If you already have the information from registration, match it up later. Otherwise, if demographic information is a must for your survey analysis then make sure to put them at the end of the survey.

7. Ask intensity questions. You don't want to just ask yes-or-no questions. You want to know the degree with which the survey respondent agrees or disagrees with your statement. For example, don't ask if you should include public speaking skills at your next training conference, instead ask how valuable the attendees would find a session on public speaking. Asking these types of questions in the pre-event survey can help you prioritize the agenda. If you get a lot of weak interest for one session, you may want to include the session that got fewer, but more passionate, votes instead.

8. Offer a midpoint. As I've mentioned in the past, there are passionate arguments among market researchers about how many points should be on your scale. This article suggests going with an odd number scale because it has a true midpoint.

9. Questionnaire design should be balanced. This is another suggestion by Eric I fully agree with. Survey design is a definite skill. Some people try to write a lot of open-ended questions when they're working on designing survey questionnaires because they think they get better data. This is a mistake. You want to have a mix of close-ended questions (multiple select, single select, rank questions) and open-ended questions. The best surveys will include more closed-ended questions. They provide you with better data and are much easier when it comes to the survey data analysis.

10. Introduce the survey. We've gone over this many times. It's important to introduce your questionnaire. In the introduction you want to explain what the survey is about and why you're conducting it. By providing survey respondents with this information, you're likely to see a higher response rate. Introductions get "buy in" from the survey sample, they see that the data collection is important and are more likely to complete your survey.

Do you have anything to add to these conference and seminar survey tips? What questions have you asked in pre-conference surveys to  help you plan the event?

This Friday we are will be celebrating our blogs' first birthday. I can hardly believe it's been almost a year of sharing our Web Survey Best Practices with you. Each day this week we will be counting down our top favorite posts. You might have already guessed, today's theme is employee surveys.

• HR Survey Tips: Weed Out Unqualified Job Candidates: In this post, we talked about how pre employment assessments can be a big time saver for HR. It can be a real struggle for hiring managers to identify the best candidates. It's no secret that asking pre-interview questions or providing job applicants with a pre-interview questionnaire can save you from having to speak with people who are not a fit from the start.

• Employee Surveys Can Help Cut Costs: This post highlights how staff opinion surveys can help you identify areas to cut costs. You may find that the best cost cutting ideas come from the trenches and not senior management. All you need to do is ask.

• Employee Satisfaction Surveys Boost Productivity: This post is about employee morale and satisfaction surveys. Research has shown that motivated and satisfied employees tend to contribute more in terms of organizational productivity and maintaining a commitment to customer satisfaction. So how do you make sure your employees are, and remain, motivated and satisfied?

• When other companies come calling, will your employees remain loyal? This post talks about how you may have high employee retention rates now, but that doesn't mean you have satisfied employees. When the economy picks up again, organizations will begin to grow their workforce and skilled professionals will be in high demand. If your business fails to take care of their employees now, they may not keep those employees once hiring starts again.

• Employee Surveys Can Improve Customer Experience: This is my favorite post on the list, probably because my precondition to focus on the importance of creating a positive customer experience. One of the best ways to measure client satisfaction and experiences is to measure employee loyalty using employee satisfaction surveys.

Just because you find correlation in your data when analyzing survey results does not mean there is causation. I find this is a common mistake in lots of survey reports when someone is new to survey research or conducting data analysis. Take this example:


Some people see this connection as a causation. Customers are satisfied because you contact them frequently to make sure everything is going well. The problem is, it's not a causation. Causation are extremely hard to prove because you cannot control every factor. For example, you may split your territory by industry and your solution suits some industries better than others. Or clients who are really satisfied simply use the product more often so the customer success managers reach out to them more frequently, because they are more likely to have questions. While those who use the product less have less to be satisfied about and may feel they are wasting those budget dollars.

To be able to prove causation, you need to be able to rule out all other possible explanations for the connection. As you can imagine, that's almost impossible to do since we do not control outside factors influencing the survey respondent or even the greater survey sample. Instead, when situations like these occur, we're seeing a correlation between two things. In my customer survey question examples, there's a correlation between how satisfied customers and how often they are contacted.

This principle does not only apply to customer survey research, it also applies to analyzing employee feedback forms, product surveys, market research and any other type of data collection and analysis.

I came across an interesting conversation today where someone was objecting to conducting a customer satisfaction survey because they were concerned the customer questionnaire would only reveal negative customer feedback. This is definitely the wrong attitude. If you think online survey results will only contain negative feedback, it's even more important you select a feedback management tool or survey designing software TODAY and create a customer satisfaction survey tomorrow. Then start fixing the problems that are resulting in all the negative feedback.

Fortunately, the person who thought they would only receive negative feedback will probably be proven wrong. Unless your organization has some serious issues you've been turning a blind eye to for awhile, no survey - customer or employee survey - is going to result in all negative feedback. If everyone hated your company, then you would have no customers and/or no employees. So relax.

The misnomer here is that only unhappy people complete survey questionnaires. This is not true. Happy customers complete them too. Even people who don't have strong opinions either way complete surveys. I honestly cannot say it enough: conducting business surveys to measure customer satisfaction or employee loyalty is essential to all organizations.

Don't be afraid of negative feedback. It's actually a good thing because it forces you to make improvements. Your organization isn't perfect, and there is always more room for growth. Sign up for a free trial today and begin collecting feedback from customers and employees.

Not too long ago I wrote about measuring customer satisfaction adding Net Promoter Score survey questions to your customer satisfaction surveys, and how you could classify clients and consumers based on their responses. I mentioned how you should use these classifications to look for similarities within the groups. This can help you identify possible segments that are best for your organization to target or possible reasons for how satisfied or dissatisfied a client is. Here are some other tips for what to look for:

Do you know how satisfied customers are? Do you know commonalities satisfied customers share? What about dissatisfied customers? If not, you're missing out on valuable information that could help identify how your organization can boost revenue. All you need is to conduct some customer satisfaction research. If it could help boost revenue why wouldn't you be interested in measuring customer satisfaction?

One way to begin to classify customers in customer satisfaction survey research is to ask a series of Net Promoter survey questions about overall satisfaction, likelihood to repurchase and likelihood to recommend. As I've mentioned in past posts, many online survey tips recommend placing overall satisfaction questions before more specific questions. This applies here as well. Asking about repurchase likelihood and chances of making recommendations should be placed where ever they fit best within your questionnaire.

Here's the important part: the survey analysis. After getting back your questionnaire responses and you begin the survey results analysis and calculate customer satisfaction scores, look at how you can divide up customers. Typically, customer survey best practices say there are four distinct customer satisfaction categories:


This should just be one step in your survey research analysis process. From here, one of the things you should look for is commonalities between each group. Does everyone in the dissatisfied column have something in common you can fix? Does everyone in the secure group do something the bottom two groups do not, maybe you could better share these benefits or features.

There are always a lot of questions about customer satisfaction survey methodology. Like with all survey research projects, the first step is to clearly define the goals of your customer satisfaction marketing survey. One of the goals when measuring customer satisfaction should be to increase customer satisfaction in ways that will maximize the value of long term customer relationships. I often see people developing customer satisfaction surveys that fail to collect data to help organizations do this. Instead, their satisfaction questionnaires simply categorize clients into a satisfied or dissatisfied pile based on the response to one overall satisfaction question.

In the past, I've heard people compare measuring customer satisfaction to measuring an individual's level of intelligence. I think it's a good example for understanding why one overall satisfaction is not a reliable means of gauging satisfaction among customers. If you are asked to take a test intended to gauge how smart you are and you open the test booklet to find a single question:

Chances are, you wouldn't think it was a good gauge of your intelligence - in fact, it wouldn't even tell the researcher how observant you are as a person. That's because how intelligent someone is cannot be reduced to a single question, intelligence is multidimensional. The same is true for measuring customer satisfaction, you cannot get a true understanding of how satisfied a client is or isn't by asking a single question.

What does this mean to you when starting your next questionnaire builder? If you select a quantitative research method, like surveying, be sure to include a series of questions aimed at gaining an understanding of customer satisfaction levels. Keep in mind your goals and how you could increase satisfaction in ways that will transfer into actionable items for the organization. When analyzing the survey data, don't just lump customers into satisfied and dissatisfied categories. You should be assigning degrees of satisfaction based on the responses.

I was looking through a customer survey template the other day and the first question was "Which of these products do you use?" The options were Product A, Product B, Product X and Product Y. Umm, hello? The point of a customer questionnaire is that you're surveying customers, which means you should already know what product survey respondents bought. Now if you're a B2C organization that distributes products through retail stores, this statement might be a little unfair since a consumer can go into any Target and purchase your product. But for now, let's assume you don't have this channel for product distribution.

Why are you making your business survey longer instead of shorter? You already know the answer, put this information in the contact record. Since you have the data, I would even recommend including the question, hiding it and importing the answers to this question for respondents before sending out any email marketing soliciting survey responses. This way, you can utilized online survey tool features like Pipe Logic. Using Pipe Question Logic allows you to personalize the respondent experience and keep them more engaged. Customer survey best practices tell us more engaged respondents are less likely to abandon your survey, aka increase your response rates.

What other common questions could you eliminate during customer research studies because you already have the data somewhere in your organization?

Seems super simple to add this example survey question to any type of online survey - right? Yet, it's a frequently forgotten question. I'll admit even I forget the need for this type of question. I'm usually focused on keeping my questionnaire as short as possible by limiting my survey questions to only essential ones.

However, when I'm filling out a survey, I typically look forward to those comment boxes so I can provide additional customer feedback. Particularly when I'm filling out a feedback form measuring customer satisfaction. For example, I was watching a TV episode on abc.com and was having trouble with their media player (I always have trouble with their media player, and they are constantly changing it!) When I finished watching and closed the browser, I received an invitation to complete a customer satisfaction questionnaire. I was expecting one of the customer satisfaction survey questions to be an "Additional Comments" box, since none of the questions centered around my viewing satisfaction. No such luck. I was seriously disappointed. I wanted to give my feedback so hopefully, next time the viewing would be better.

Most people agree to fill out your survey because they believe there's something in it for them. You don't have to offer an incentive to make people think there's something in it for them. Simply asking for customer or employee feedback implies that you're ready to make changes, that you want to identify problems and fix them.

If you fail to offer a "last ditch effort" with and additional comments (optional) question, you could be making a huge mistake. Depending on your survey design, it may be the only place a survey respondent can give you truly honest, unprompted feedback. While you can craft technically perfect questions, they still limit what you can find out from questionnaire respondents. The additional comments section may bring to light something that wasn't even on your radar: new product enhancement requests, process changes that ultimately could boost employee and customer retention, customer service issues, website design flaws, etc. Some organizations have gotten great ideas to improve their retention programs from consumer feedback.

Including an additional comments section is even more important with online surveys, because with telephone or paper surveys the respondent has the chance to at least tell the interviewer or write in extra answers. It's always possible the extra comments may be discarded, but the respondent can still try to give additional feedback. There's no extra chance with a web based survey. Adding this one extra question wont destroy your response rate or cause your survey to be bad. In fact since it's such a common question type, I would guess if the respondent doesn't have additional comments, they hardly even notice the question.

We recently added two new question types to our web survey software: Matrix Spreadsheet and Net Promoter Score. If you're not familiar with the Net Promoter Score, here's some quick background. Introduced in 2003 in a Harvard Business Review article by Frederick Reichheld, Net Promoter simplifies the complexity of measuring customer satisfaction. The Net Promoter Score gives a stable measurement of performance that can be compared across the organization or industries.

An organization can obtain their Net Promoter Score by asking customers a single question on an 11 point scale (0-10). Questions typically ask how likely the respondent is to recommend a product or service.


Based on the response, respondents fall into one of three categories: Detractors, Promoters or Passives. Detractors answer with a 0-6 rating. Detractors are unhappy customers who could potentially damage your brand through negative word-of-mouth. Passives answer with a 7 or 8 rating. Passives are individuals who are satisfied but are vulnerable to competitor offerings. When calculating the Net Promoter Score, Passives are ignored. Promoters answer at the top of the scale with a 9 or 10 rating. These people are loyal customers and brand enthusiasts. Promoters will continue to buy your products and refer others, ultimately fueling an organization's growth.

To calculate your Net Promoter Score, simply subtract the percentage of detractors (those who answered with a 0-6 rating) from the percentage of promoter respondents (those who gave a 9 or 10 rating)


The goal is to have a high Net Promoter Score. Typically, 75% is considered a good Net Promoter Score. We recommend using an open ended question after a Net Promoter question. This allows appropriate action to be taken by employees and management to identify and resolve customer issues.

Sometimes people think good customer service automatically equates to satisfied customers.  But customer service is just one factor contributing to customer satisfaction and experience.  Research shows customers switch to competitors, become repeat customers, and recommend products and services based on their overall satisfaction experience, not just customer service experiences.  This should seem pretty intuitive.  Customer satisfaction can be grouped into three categories:

• Dissatisfaction.  Customers falling into this group will not make future purchases and will probably share their negative experience with their network (costing you future customers), complain vigorously or take legal action.  If you find your experience scores fall into this category, you are probably providing bad customer service and falling short of other satisfaction categories as well.

• Indifference.  If you're seeing satisfaction survey scores fall in the indifference category, customer service feedback should be excellent.  However, other factors such as value, quality, environment, ease of use, etc. could be pulling down the overall measure of customer experience.  Good customer service can do a lot to move customers from dissatisfaction to indifference, so if your product is difficult to use having a great customer care team could save you losing customers forever.

• Loyal. Customers who are loyal find that all factors contributing to their experience with your organization are exceeding expectations.  The more these customer interact with your organization, the more loyal they become, increasing brand recommendations to their network.  Organizations with satisfaction scores falling into this group are considered "world-class" with strong reputations.

Measuring customer satisfaction is an important exercise for any organization.  Not only will it tell you where you stand now, but if you're ranking in the dissatisfaction category, you could make changes to keep customers from leaving. So what is your customer satisfaction score? 

Last week, we blogged about how conducting a survey creates expectations, whether it's a staff opinion survey or a customer service feedback form. Regarding this topic, Seth Godin's recent Direct from Consumer Marketing post caught our eye. 

In this post, Godin asks if organizations really want to hear from customers and clients—particularly if they're unhappy. If you're in the business of selling something, we assume you would respond the way Godin anticipates: of course you want to hear from them!

Still, actions speak louder than words. If you conduct a survey and ignore the responses, you send the message that you're not interested in what customers have to say.

Beginning a conversation with a client is sometimes the hardest part. Online surveys to measure customer satisfaction or gather product feedback are an ideal way to open the lines of communication and learn more about your customers. But you have to analyze the data; you have to act. You have to listen. Meet the expectations you create through a survey campaign and listen to your customers, and you'll increase customer loyalty and customer retention. 

We recently came across an article in Customer Service Magazine that pointed out that your customers are not just comparing their experience with you to your competitors, but to any outstanding experience they've had with any company. This should prompt you to wonder, How does your customer experience stack up to organizations such as L.L. Bean, which recently ranked number one in retail customer service?

Customer service should be important to every organization because it links back to customer retention and customer satisfaction. Using customer satisfaction surveys is a great way to find out how your customers feel about your service and their experience. To find out how measuring customer satisfaction can lead to increased customer satisfaction and retention, sign up for one of our best practice webinars.