Market Research

I always recommend probability-based survey sampling techniques wherever possible.  Sometimes, however, companies and organizations want to get an initial feel for how consumers and customers will react to a new product or concept.  In addition, early in the process, you may not have the ability to comprehensively identify a target market or sampling frame, and there is no way to produce a representative sample of your population. 

In these instances, it may be useful to employ a snowball sampling technique as a pilot project, or to gain a rough, early grasp on what customers are feeling.  A snowball sample is one in which you use an initial group of respondents as recruiters for additional market research respondents.  In the survey, you ask your original respondents to list several people  they know that might be interested in completing a survey as well.   This is a case where an incentive might prove particularly useful, since you are asking your market research survey sample to provide contact information of their acquaintances.  Snowball sampling is also especially useful if you do not have a predefined list of people to survey, or if you are trying to identify key information-holders or opinion leaders.

Again, there is a significant caveat of snowball and other nonprobability-based business research methods for sampling techniques: they do not produce representative samples, and therefore cannot be used to generalize findings to the overall population.  But if you are just starting out, and do not mind that your market survey sample cannot produce generalizable findings, then a snowball sampling technique is a convenient survey data collection method to obtain larger amounts of preliminary data.

Market researchers write survey questions for different audiences all the time. Sometimes the groups are broad, like consumers or non-consumers, and sometimes they are far narrower, like employees at a small advertising agency. When writing surveys for any audience, be sure and use the clearest and most sensible language to communicate with that audience.

For instance, if you were hired by the small advertising company to complete a 360 performance survey, you should use the correct title structure for supervisors and employees at the company. You don’t want to refer to managers, if at the agency supervisors are referred to as account executives. It would be confusing and yield improper results for your employee questionnaire.

Similarly, if you were performing a bank customer satisfaction survey, you wouldn’t want to ask survey respondents who only have standard checking accounts about their habits with their savings accounts.

In addition to doing the research necessary to communicate with your audience, you must also use clear language. There’s nothing worse than trying to respond to a survey questionnaire that is poorly written, with confusing grammar or overly long sentences.

Writing as clearly and directly as possibly will give you the answers you need. Have multiple people proofread your survey if possible. Everyone processes information differently and a variety of people may find different errors or points of confusion. Having others read the survey is beneficial, because as the survey creator it is hard not to be biased. You need to make sure it is clear to the respondents and as the survey creator, it is hard not to be biased.

When many of us think product market research, we think along the lines of consumer interest or consumer satisfaction studies. But what about a study conducted specifically to find ways to improve a product?

Sometimes a client may want to find ways to improve an existing product or service. It is risky to launch a brand-new product, particularly in today’s economy when businesses are failing more frequently. It is often better to grow a successful product than try and start from scratch.

When you are conducting a study for this purpose, it is important to analyze from several angles. You need to gather current consumers for a consumer satisfaction survey, as well as non-consumers for a survey or focus group. To gather product feedback ask the consumers questions like:


Utilize the survey to gather their opinions on pricing, quality and more. The client can use this information about what they are already doing right as a baseline for improving their product.

The non-consumers are a little tricky. You need to find out why they don’t purchase or use the product. Maybe the price point is just a little too high. Depending on what type of product it is, perhaps the client might be able to offer a similar item at a lower price point to gain more customers.

Perception of non-consumers is just as important as that of consumers. Perhaps the reason they are put off is something the client can easily fix, thus gaining new and loyal customers.

When performing a market research study for clients in certain marketplaces, it is important to remember the competition. If, for example, you were doing a customer market research survey for a fast-food restaurant, Burger One, you would want to gauge survey respondents’ feelings about the competition as well as your client.

You might ask restaurant customer satisfaction questions like:


If respondents frequent Burger One and four other fast-food restaurants, that gives you insight into the survey data provided. If respondents only frequent Burger One, that is helpful to know as well. Take this example of how survey results can be applied to the restaurant's over all marketing strategy:


On the other side of this equation is what Burger One's loyal customer base thinks. If customers who only eat fast-food at Burger One express no interest in the new sandwich, it may not be the best possible sandwich to launch. However, if respondents who frequent competitors more often than Burger One express high levels of interest in the new sandwich, it could spell an opportunity for Burger One to gain new business.

It’s best to analyze market research from all angles when launching a new product – there are multiple factors at play.

Does your customer survey have "survivor bias"? I'm betting it does. Why? Because when most survey designers create business questionnaires or client satisfaction surveys we only collect feedback from individuals who are still customers. What about lost business? Why aren't those customers who left asked to complete your client feedback form? You better believe they have feedback.

In her post at the Dimensional Research blog, Customer Satisfaction Surveys: Avoid "Survivor Bias", Diane Hagglund defines "Survivor Bias" as drawing conclusions only from data that is available or convenient and thus systematically biasing your results. AKA biasing your survey sample by only asking "happy" customers. They may not be over the moon about your product, but they're at least happy enough with your offering if they're still paying you.

I agree with Diane on this one, it's pretty ridiculous to allow this bias to creep into your customer market research. It's easily avoided considering you should have all the customer data you need to send them the same client survey template. Make sure you're measuring client satisfaction among customers who left you for a competitor as well as those who simply decided not to renew the service (and didn't go with anyone else).

When you begin writing customer survey questions to create client questionnaires or update your annual template for a client satisfaction survey make sure you think about your former customers. If you don't know who they are, this is a good time to find out. Thank goodness for internet research software and customer insight solutions to make the act of surveying clients a little easier.

As market researchers, we are all aware of the many ways data can be gathered for a study. There is the online survey, telephone survey, in-person interview, direct mail questionnaire, email survey, focus groups and more.

Telephone surveys often yield decent response rates but can be difficult to conduct. A lot of people simply don’t want to be bothered on the phone and spend time answering poll questions. But phone surveys are great because you can ask all the questions and get clarification on any answers that don’t quite have the information you are looking for. The opportunity for that is of course not present in an online, email or direct mail survey.

A telephone survey is not going to work for every study, however. There are some products that won’t match up well. For example, a survey about a medical product is likely not one that anyone will want to discuss over the phone with a stranger.

For a consumer survey with a lot of feedback answers using numbered responses, an email questionnaire might be best as it could grow tedious and leave too much margin for error.

If it works, you can also combine survey methods and gather data both ways. When creating the surveys be sure to incorporate the differences into the call script and email. If it is necessary to have pre-screening questions for the phone interview, make them as concise as possible. Combining data collection methods is helpful and can achieve a more well-rounded result, but be careful about choosing the best method for your purposes.

When working with a client who has already built a strong customer base, market researchers are often called upon to create surveys to aid with customer retention programs.

Customer retention surveys are a bit different than a new product or service launch survey because the survey respondent sample must consist of product users. There are several things you need to ask this group so your client can formulate the best customer retention strategies, including:


The recommendation sample customer questionnaire question is important. Often, this can be a company’s bread-and-butter when it comes to retaining customers and gaining new ones. If a good amount of current customers would recommend the product to a friend, that speaks volumes about the quality.

It is also wise to include some open-ended questions. Find out why people use the product, and would continue to do so. If they wouldn’t recommend the product, or use it again, it is important to know why. Your client can use the information to persuade customers on the fence, and keep current customers coming back.

Before creating a poll online for your client survey, sit down and gauge initial opinions and move forward from there. If there were original product launch studies done, it may be helpful to revisit those results as well, just for comparison purposes. Approach the questionnaire research with care and the resulting data analysis will provide information to create customer retention techniques and marketing strategies.

Increasing survey response rates is a major goal of most survey builders and market researchers. There's an art and a science to increasing campaign response rates whether it's an email marketing campaign or an online market research study. I wanted to share some of my tips for how to increase survey response rates:

• Make the email survey invitation from names easy to recognize. You can do this by including an individual's name within the organization that's well known (such as the CEO or if it's a client survey, the name of their sales rep). You can also use the organization's name, or both. For example, I'm subscribed to a few MarketingProf's newsletters. When they send out emails they include the same person's name and their organizations name so it looks like this: Anne, MarketingProfs. I recognize it everytime, and since I enjoy their newsletter, I made sure to open the email.

• Keep subject line's compelling, but short. The subject line and the From Name are the two most critical pieces to get your email opened. Try to keep your subject lines to 35 characters or less. You don't need to put the entire email in the subject line, but you do need to include enough information to make the recipients open the survey email.

• Create an attractive survey invitation. Studies have shown that well done HTML email messages get better response rates than plain text emails. With HTML you have the opportunity to include images, change font sizes, bold text, etc. Take advantage of this chance. It's one more way to get people to click through to your survey!

• Send personalized survey invitations. Personalizing your emails, even something as simple as including the recipient's name in the greeting, will return a higher response than a generic message. It creates a personal touch, and makes the recipient feel like someone took the time to send them a personal message (even though your email survey tool did it for you).

• Introduce the survey. Let the participants know why they should participate in your survey. If they don't understand why their opinion is important to your survey findings, why would they want to take the time to fill it out?

• How long will the survey take? Not setting expectations in the beginning for survey length leads to low response rates and high abandonment rates. Not what you want to see. If you don't let people know how long it will take to fill out your online survey, they're going to assume you're hiding something about how long it is. Tell them it will take X minutes or the survey is only Y questions long. Definitely be honest, if you lie here, you're going to hurt your future chances of getting those respondents to complete your next online survey questionnaire.

• Remind your survey sample that their responses will be kept confidential. This is particularly important for surveys about uncomfortable topics. For example, you created a poll for a public opinion survey to see how your population feels about an emotional topic such as abortion. If you don't keep the information confidential, you probably will not get honest feedback. Same thing goes with employee questionnaires, they should always be kept confidential and anonymous. Not keeping responses confidential will definitely hurt your response rate, as well as the validity of the data.

• Offer an incentive. Offering incentives is a proven method for increasing survey response rates. But this method doesn't work if you don't let people know about it up front. Put it in your email, put it on the welcome page of the survey, then make sure to follow up. Again, if you drop the ball here, the chances of that survey respondent completing your online survey form in the future is drastically reduced.

• Always say Thank You! Remember when your mom always made you write thank you notes when you got presents or cards from people for holidays and birthdays? It was because people like to feel their effort is noticed and appreciated. Same idea here, it's nice to just get a short note thanking them for their time. After all, survey respondents are doing you a favor.

• Don't over email your contact list. This is very important. You shouldn't be inviting the same people to complete your online surveys every month. It's important to segment your list using whichever survey sampling method that works best for your surveys to avoid email list fatigue. Make sure that you're coordinating with other campaigns as well. Just because you know they're different initiatives you're emailing about, your contact's wont necessarily see it that way. If you begin sending emails too frequently, contacts are going to just delete your email and never open it, opt out or report you as a spammer.

Have a tip that I missed? I'd love to hear it!

A good survey includes both qualitative and quantitative data, but when gathering qualitative data it is important to gather the right kind of answers. Go too open-ended with your question, and you may end up with answers that do not help you with your study at all.

Consumer insights into a new product or service are important to a market research study, and open-ended questions are the way to get them. When conducting consumer surveys, there are several online survey tips you can follow to get the right mix of closed and open-ended question:

1. Determine if it is a question that you cannot ask in any other way. For example, if you were asking consumers how often they purchased a product, it would be difficult to analyze data if that question were left open-ended. Group responses into categories like once a month, twice a month, more than three times a month.

2. If the question must be open-ended, make it narrow. Sounds contradictory, but if you ask too much, you’ll get too much. Don’t ask if they like your product and if they share it with friends in one breath. Ask simple questions like: Why do you purchase the product?

3. Allow expansion on closed-ended questions. If you are unsure about something, leave room for comments on close-ended questions. This is a good compromise that can help you get the responses you need.

These are just a few best practices for survey writing, but it goes without saying that when creating a survey, you need to put as much, if not more, effort into writing the questions as you will analyzing the data.

I’ve written in the past that it’s best to use probability-based sampling techniques for your employee, market research or consumer survey samples whenever possible.  One of these techniques is called stratified sampling.  It is used when you expect that your population is heterogeneous (different) and that the survey results could vary greatly based upon certain subgroup characteristics.  If you are particularly interested in subgroup analysis within your survey sample, using a stratified sample can increase the accuracy of your results and reduce error estimates.  Here’s how it works:

Let’s say you are interested the television watching habits of American citizens, and you know (from anecdotal evidence or previous theory) that television watching varies widely by gender.  Since you know that males and females have disparate television watching habits, you want to select a sample whose results will accurately reflect the habits and responses for males and females independently.

What you do is first divide the population into mutually exclusive subgroups (or “strata”), and then take a random sample from each of the subgroups.  Using our gender example, you would divide the population into two groups (male and female), then take a random sample from your male and female subgroups, respectively.  You will be able to be more confident in what you say about your results than if you used a simple random sample from the overall population.

I was asked to complete a product market research survey today. The product survey asked me a series of questions about possible products related to improving my beer drinking experience. One of the many things I didn't like about this survey was they never asked me if I thought there was a problem.

For example, one of the possible products was for a different method of carrying beer. Unfortunately, I'm perfectly happy with my current method: keeping the beer in the package it comes in. The suggested product would make me move the bottles from their packaging to take them somewhere. I'm not interested in that extra step. Did they ask me why I wouldn't by the product? Nope. They only asked if I would.

Maybe for their survey research goals, that's all they cared about: Would I buy the product, Yes or No? Kelli mentioned in a post earlier today that it's important to gather product feedback on a smaller scale before launching a bigger market research study. The goal of this market research questionnaire easily could have been to identify which product ideas should be considered for further qualitative and quantitative research. Only products where more than 20% of survey respondents indicated they would be interested in purchasing will have more research around it.

On the other hand, I like to make sure my position is clear when completing attitude surveys. This is probably a respondent bias I bring to the table because I spend so much time sharing online survey best practices. I know that often survey questions are written in a way that doesn't always collect the data the survey creator wants. They just don't always write the right questions. So avoid being "mis-quoted" so to speak, I like to be able to answer a series of question about why I feel one way or another.

Believe me, there were a lot of poorly written survey questions in the feedback form, as well as other issues with their data collection method. I've found that where there is one serious problem with a survey template, there are others hidden as well. If you're worried your survey questionnaire may have problems, go ahead and ask our online survey Professional Services Group to take a look. We're always happy to help.

Looking for sample market research survey questions? Here are a few sample product survey questions you can use in your next product research survey:

Market research is incredibly useful at any stage of the game. Though starting out small may seem like an unnecessary step, sometimes it works out better for you and your client. A thorough study is necessary for any new product or service launch. When dealing with a NEW concept, you may want to conduct a smaller initial study to gauge interest level before continuing.

When conducting the smaller study, set criteria for continuation of the study. Criteria for every client and every study will be different, but the goal of setting parameters should always be the same. Decide what percentage of respondents must be interested in order for you to further develop the concept before continuing the study.

For example, if you gathering product feedback for a major cleaning product company about a brand-new product, you might want 60% of product market research survey respondents to express an interest in order to develop the study further. A full product market research study is extremely time-consuming - the entire project will likely include multiple online product surveys, focus groups and other data-gathering methods.

Market research projects like this have a lot of moving parts - your team will have to write survey questions, compile results and create presentations based on their market research analysis of the data. If it turns out that only 20% of respondents are interested in the new product, it may be better to return to the development phase before continuing with the study.

Even though it might be tempting to forge ahead, particularly in lean economic times, your clients will appreciate you more if you save them time, effort and money by starting out small.

The demands of a market research study are complex. You may need a large number of respondents, but all those respondents need to be the “right” respondents. You don’t want survey responses from outside the target demographic skewing your data, forcing you to start over.

For example, a survey questionnaire we did at my company was recently returned with highly skewed data. We were surveying a school where 78% of the student body was composed of “in-state” students. When we got the results back, only 3% of respondents were in-state students – definitely not representative of the population.

Where we erred: We did not specify what percentage of different groups of people the survey should be sent to, and did not set up quotas to get a certain mix of people. Instead, we just let it fall out naturally, which should have returned results representative of the population.

It is important to specify the type of respondents you need and decide who the email survey will be sent to, and if necessary, set up quotas to ensure a respondent set that matches the population.

It would be almost impossible to cover all the survey pitfalls you can encounter while sampling. My best advice would be to use common sense. When starting the study I’d advise doing a “soft launch” (sending out a small amount of sample) and then check the returns.  If you find that the completes from the soft launch are not representative, work to iron out any issues before doing a full launch.

When performing a market research study, researchers too often hone in on target demographics, current customers and their loyalty, price and other concerns. These are all important and should not be ignored. However, we must also remember to include the competition in our analysis for a well-rounded marketplace.

Your client will want to know who is out there in the marketplace, how much of the market they’ve cornered, whether or not there is room for their product and more. Sometimes, a market can have room for any number of competitors, and sometimes the competitive analysis may reveal that there is no room for a new product.

However, if the client decides to forge ahead with a new product launch, a good market research study can be used to help them come up with a well-planned strategy for overcoming the competition. Find out if respondents are loyal to a competitors’ brand, and why. Ask questions like:


Answers to these questions will help your client figure out what the competition is doing right and wrong. This will enable them to create their own unique brand identity that incorporates some of the positives from the competition as well as solutions to any problems.  While the bulk of a market research study, survey or focus group should relate specifically to the clients’ product, it is key to include the competition for a well-rounded, thoughtful result.

Yesterday, I discussed a simple way to establish a Facebook group for market research uses. Today I will outline some easy ways you can use the group to aid you in market research.

Now that you’ve got your Facebook group established, be sure to update it fairly frequently to keep participants interested and engaged. If the group relates to an established product line, include updates and information on the line, promotions and developments relating to the line.

In keeping with the running shoe market research example from yesterday, you might include links to articles where the running shoe is featured or mentioned, marathon or race tips, and the like. Also post items for response on the discussion boards for the group.

You can even begin a pre-screening process for a survey or focus group on the discussion boards. Post a call for participants and a link to an online pre-screening survey to determine if respondents qualify for your poll or focus group. Facebook group participants will provide an interested, more enthusiastic audience as they are already invested in your product or company, but you still must be careful to ensure you get the right target demographic for any official studies.

You can use the group in a less official capacity to get qualitative data and “buzz” about the product. For example, if the running shoe company launched a new product six months ago, you could ask for general feedback on the product – who is using it, their thoughts, etc. The responses will be helpful when creating survey questions later.

Facebook is an inexpensive and not very time-consuming way to get additional customer feedback and potential survey participants. If you exercise caution, you can really make it work for you.

The social media storm has hit and continues to rain down a variety of opportunities for market researchers. Everyone’s getting on Facebook, Twitter, Yammer, and the like.

What does this mean for market researchers? Social media sites can provide another avenue to explore for focus group participants or survey respondents. In fact, they make it easy for you.

Think about a typical Facebook user page, for example. Most users generally input their age, location, high school and college graduation dates, and some basic personal information. Some are even more elaborate, putting their job title, functions and a veritable fact sheet on their life and interests. Essentially, a Facebook user page is a sort of consumer profile.

The Facebook ad system allows you to target particular groups. For instance, if you were doing market research for a running shoe, you could target those who list running as an interest in their profile, in a certain area or age group – there are many possibilities. Invite them to join the group and gather new product feedback through your ad.

Once you have your members, you can view their basic information to see who fits your target profiles. You can also utilize the discussion pages or Facebook “wall” to gather general, qualitative remarks about the product type. While Facebook is a good place to start building a target group, naturally you must work closely with your client about what you can reveal online, and exercise caution in what you reveal to the group.

Check back tomorrow, when I will discuss additional ways you can utilize the Facebook group to send out surveys and gather information.

Yesterday, I shared the first 5 tips from the Meetings & Convention article, Survey Science: How to craft more effective attendee evaluations. No one can deny the importance of measuring customer satisfaction, so why wouldn't you want to conduct a post-event survey to measure attendee satisfaction? Particularly if it's an annual conference, meeting or tradeshow. If you want attendees to keep coming to your events, you need to make sure you're gathering feedback on what worked well, what didn't, what should be included next year.

In continuing with the fifteen survey best practices shared in the article, here are the next five:

6. Be careful with demographic survey questions. This tip comes from our very own Eric Eden, VP of Marketing here at Cvent. Survey respondents don't always like to share all their demographic information on surveys. If you already have the information from registration, match it up later. Otherwise, if demographic information is a must for your survey analysis then make sure to put them at the end of the survey.

7. Ask intensity questions. You don't want to just ask yes-or-no questions. You want to know the degree with which the survey respondent agrees or disagrees with your statement. For example, don't ask if you should include public speaking skills at your next training conference, instead ask how valuable the attendees would find a session on public speaking. Asking these types of questions in the pre-event survey can help you prioritize the agenda. If you get a lot of weak interest for one session, you may want to include the session that got fewer, but more passionate, votes instead.

8. Offer a midpoint. As I've mentioned in the past, there are passionate arguments among market researchers about how many points should be on your scale. This article suggests going with an odd number scale because it has a true midpoint.

9. Questionnaire design should be balanced. This is another suggestion by Eric I fully agree with. Survey design is a definite skill. Some people try to write a lot of open-ended questions when they're working on designing survey questionnaires because they think they get better data. This is a mistake. You want to have a mix of close-ended questions (multiple select, single select, rank questions) and open-ended questions. The best surveys will include more closed-ended questions. They provide you with better data and are much easier when it comes to the survey data analysis.

10. Introduce the survey. We've gone over this many times. It's important to introduce your questionnaire. In the introduction you want to explain what the survey is about and why you're conducting it. By providing survey respondents with this information, you're likely to see a higher response rate. Introductions get "buy in" from the survey sample, they see that the data collection is important and are more likely to complete your survey.

Do you have anything to add to these conference and seminar survey tips? What questions have you asked in pre-conference surveys to  help you plan the event?

Many people ask the question How do I create a customer survey? The basic steps are the same no matter what type of client survey you're writing: customer satisfaction, global market research, product development surveys, etc.

Step 1: Sit down and figure out what the goal of the survey is. Are you trying to identify upsell opportunities? Want to discover features missing from your current product? Figuring out if it's a good idea to take your marketing overseas and attack a global market? In the very beginning of the survey planning process, you should know what it is you want to get out of the consumer survey. If you don't have a firm customer satisfaction survey objectives in the beginning, while you go through the other steps such as writing survey questions or selecting the best survey software, you're going to stray from the path. If you stray from the path, you may find the final survey results are not as helpful as you had hoped.

Step 2: Decide on a research methodology. Your goals should help you on this step as well. You need to first decide if you're planning to do qualitative or quantitative research. From there narrow the scope further, if you want to do qualitative research are you interested in focus groups, advisory boards, one-on-one interviews? With quantitative research you may decide on comment cards, feedback forms and surveys. Is your survey method going to be online, telephone-based or paper questionnaires?

Step 3: Survey Design. I'm making the assumption since you're reading a survey blog about how to create customer surveys, you're not interested in the other market research methods right now so I'm going to focus on the process of building customer surveys. Once you've gotten through the first two steps, you're ready to start writing survey questions (Finally! I bet you thought this would be the first step!). Customer satisfaction survey design can be the biggest challenge. Luckily, there is survey designing software to help you step through this. Survey software tools often have templates and question libraries to help you write good survey questions.

Step 4: Data collection. Okay, you've picked your customer survey methodology, created a client survey and you're ready to field your survey (or use the data collection tool in your survey application to collect responses). Exactly what you do in this step will depend on what type of survey you decided to collected: telephone, paper, online. One way to get survey responses is to use email marketing tools to send personalized email surveys. You can also share the link on your website, social media sites, invoices, etc.

Step 5: Analyze customer feedback. Analyzing survey data is one of people's least favorite parts of the surveying process. We have some tips for how to analyze survey data here. Don't be afraid of this step. You need to conduct the survey customer analysis to achieve your goal. It's what you set out to do, so keep your chin up. You're only a step away from the final product (and once you choose survey analysis methods you should be almost finished).

Step 6: Share the survey findings. This is what you set out to do. Get answers to your customer questions. Take the customer feedback analysis you completed in the last step and format it. You're creating a survey report you can share within your organization (and maybe with others outside of your organization). If you need tips for creating survey reports or an example survey report, you can read more about them here.

Step 6.1: Take action. This is still part of step 6, but it's important enough it should be broken out. In your customer analysis survey report, you should have shared your recommendations for moving forward. Make sure you make recommendations and there is an agreement about moving forward based on the customer survey findings. If you don't plan to take action in Step 1, then you should save yourself the time of conducting the customer research in the first place.

Any other survey research design tips? How have you used these steps to create a customer survey that improved processes in your organization?

As is the case with validity, there are multiple aspects of reliability in survey research.  In general though, while validity refers to accuracy in question design, reliability refers to the consistency of your results (i.e. the probability of producing the same results after repeated measurements). 

One way to easily conceptualize reliability is to consider it as another word for precision.  Let’s use archery as an example: if you shoot ten arrows at the target and they all hit the same area, your shots can be said to be reliable or precise. 

You can test for and estimate the reliability of your survey questions in a variety of ways.  You could use a test-retest design, where you use the same questionnaire more than once to see if the results remain consistent.  You could also send your feedback form to different survey samples (consisting of similar respondents) and then compare your results for any fluctuations.  Also, if you create a set of customer service questions designed to provide you with a composite scale, you would want the individual questions to produce similar results if they are intended to measure a single concept (an overall customer service scale, for example). 

It is also important to understand the relationship between reliability and validity; results can be reliable without being valid.  If we use our archery example, the shots are reliable, but if they aren’t near the bullseye (an undesirable outcome!) they cannot be said to be valid.  Reliability is necessary for validity, but it is not sufficient alone.  You want to accomplish both objectives: shots that land near the bullseye consistently.  By creating reliable, valid questions for attitude surveys, online market research, employee reviews or public opinion polls, you can be confident that your results are not due to chance.

When a product fails, there could be many reasons – poor economy, too much competition, or a lack of market demand. Famous examples include “New Coke,” Crystal Pepsi or Sony Betamax.

These products all failed hugely and publicly. While the advertising was blamed in many cases, it’s also true that at least part of the fault lies with poor market research. If proper focus groups and quantitative research surveys had been conducted for any of these products, the embarrassment and monetary loss for all three companies could have been avoided.

In focus groups for New Coke in the '80s, the samples of New Coke were well-received. However, participants were also satisfied with current Coca-Cola offerings, and some had strongly negative reactions when asked if they would buy this new soft drink if it replaced Coke Classic. The survey was less negative, but if the focus group results had been better used to create the quantitative research questions, things could have gone differently. 

For example, the consumer survey could have contained questions such as:

1. Are you satisfied with the original Coke formula?
2. Do you regularly purchase Coke products?
3. Would you consider buying a “New Coke” product if the flavor was different?
4. Would you buy the “New Coke” if the new formula replaced Coke Classic?

These types of questions might have given a more well-rounded view of public sentiment, like the response that happened in the focus groups. Although it might seem less exciting to have a negative result to a marketing research survey, overall the company will save money, and avoid negative press. Paying for a study is cheaper than launching a product no one wants, after all.