Online Market Research

We all know the debacles that can happen when clients and companies do not do proper research, or simply ignore it. Take the New Coke disaster, for example. While New Coke beat Pepsi in taste tests, Coke failed to ask a very key question – Do you want New Coke to replace Coca-Cola?

Instead, they went ahead and launched the new product and replaced Coca-Cola. Consumers were furious! The cost of this mistake has never been totally revealed, but Coke undoubtedly lost close to a billion dollars or more.

How can you help your market research clients avoid such costly mistakes? They have already taken the right step by hiring you to do their market research. But as the old saying goes, You can lead a horse to water, but you can’t make him drink. To get your clients to actually use your advice, make sure it’s simple to understand. You can create action items based off the survey results. For example, if your research for a grocery store chain shows that 90% of customers want more prepared food options, they can expand their prepared offerings fairly easily.

Follow up with your clients as part of their market research package. Do a smaller survey after changes have been implemented to see if they are working and being implemented well. Work with the client to create the survey and examine it from every angle, especially when launching new products. It might take time but in the long run it will save money.

We think our clients should be honored we asked for their feedback, that we singled them out to send personalized email survey invitations. From the survey respondent perspective, that makes me laugh. I see it the other way around, you (the company) should be honored I opened your email and took the time to provide you with customer feedback.

So how do you reconcile these two opposing views?

Sometimes it's necessary to offer an incentive to survey respondents. We don't always like it. Once we realize it's a necessary evil, we need to figure out what the right incentive is. There are lots of different schools of thought on survey incentives. But I want to remind you that even if you're conducting a B2B market research survey or business research survey for clients, you're still talking to consumers. Just because offering an iTunes gift card isn't very "business-y" chances are your clients will still be interested in an iTunes gift card for their time.

I'm not saying iTunes is necessarily the way to go for your organization's next survey, but it is important to remember what's going to encourage people to complete your survey isn't always going to be something you view as "business-y." Some survey participants only need the promise of getting the survey report sent to them after the survey is fielded. Others need the promise of a discount on their next order. Some survey respondents just want to be entered in a drawing for a bigger prize (Kindle anyone?). You can come up with your own incentive, just don't lose sight of the fact it needs to be a reward in the eyes of your survey sample to get you the responses you want.

Okay, this is probably nitpicking, but I’ve had an on-going debate with my colleagues as to what best describes our profession. Are we “market” researchers or “marketing” researchers? What is marketing research compared to market research? What I’ve found is that the majority of us (not me), and those that characterize our function, label us as marketing researchers, not market researchers. However, I tend to disagree with that description. It may be that I don’t want to limit what I am thought to be able to do.

The marketing mix incorporates the 5 “Ps”: 


As you know, product refers to the physical offering – function, appearance, etc. Of course services have many of the same attributes and can be categorized similarly. Price includes things such as cost, profit, discounts, etc. Place refers to getting the product/service to the consumer – distribution concepts. Promotion refers to the communication that occurs in the process of selling to consumers. Finally, people address the concepts behind who is going to purchase your organization’s offering.

Please forgive the return to Biz 101, but it does relate to the difference between a market researcher and a marketing researcher. A marketing researcher conducts studies to answer questions and solve problems involving product, price, place, promotion and people. So when someone asks, what are marketing research objectives? They usually fall into one 5 Ps. 

Is that all that we do as market researchers? Is that what we usually do?  I don’t think so. 

Those of us in the research profession also produce a considerable amount of forward looking research. One could argue that the 5 Ps can be perceived as forward looking, but when you look at it, the questions and problems behind the 5 Ps usually address products and services already in existence, or at least being contemplated. Market researchers also work on problems to address wants and needs that are NOT currently offered or being developed by an organization.

The bottom line, based on how we define market research and marketing research, is that we market researchers can do more for our organizations than marketing researchers. My apologies to marketing researchers everywhere!

You’re on the right path if you’ve decided to conduct online surveys for your business. Doing survey research can yield priceless business intelligence and information. Using online survey research software means you can collect feedback faster while aggregating survey results more efficiently.

It’s not enough to simply do web survey research. A survey for the sake of a survey won’t yield any useful data. So take steps to ensure that your web survey research pays off with data you can use, whether you’re measuring employee loyalty or researching customer satisfaction.

Make sure you know why you are conducting this web survey research before you starting writing survey questions or selecting a survey research tool. Make sure you know the answers to these pre-survey questions: What do you hope to learn? What do you hope to achieve by doing this online survey? How will you measure the data you get? Start with the end in mind and write web survey questions from there. Starting with your market research objective will help you stay focused when creating your online survey.

All market researchers are familiar with the concept of prescreening and rescreening for focus groups. But similar methods can easily be applied to online surveys and questionnaires.

When finding respondents for an online market research survey, you should first send out a prescreening survey. These should be short in length and allow you to gain the demographic data you need in order to determine if the person fits the respondent sample profile. If they are a match, they will then move forward to the full online survey. How can you rescreen them? Well, typically at an in-person focus group, respondents are re-screened upon arrival to make sure their answers match. 

For online surveys, the methodology has to be a little different. If the respondents meet the qualifications for participation, allow them to take the full survey. Then at the end of the survey, ask similar questions to the prescreening so you can determine if the responses match. It is better to not immediately rescreen respondents because they may have just taken the prescreen survey and can easily recall their answers. If they are not being truthful, this allows them to continue the lie and may leave you with skewed data. Delaying the rescreen enables you to get answers that are more truthful. Be careful with rescreening at the end of the full online survey questionnaire, it can be annoying to be asked questions they've already answered in a same session.

To complete our market research defined discussion of survey sampling errors, we will look at response errors. For those that have read the previous postings on this topic, you have seen how sampling errors are comprised of both non-response errors and response errors. Further, we looked at non-response errors resulting from unintentional exclusions in the market research sample frame or explicit decisions by those contacted to not participate in a survey. Response errors, on the other hand, arise from people taking the survey but the resultant answers are incorrect.

There are generally three types of response errors:


Measurement error results from the survey research instrument itself. Ambiguous and confusing questions can lead to respondents providing information that they believe is true but is in fact not true. They may lack an understanding what the surveyor intended to ask. This can be avoided by making sure that questions are clear and easily interpreted. In interviewing instruments, clear instructions and rigorous standards of interviewing will help alleviate measurement errors. In questionnaires, proper grammar is often the key. One way to mitigate instrument problems is make sure to pretest, whether it’s a questionnaire, focus group script or interview guide.

Recording and analysis errors are a matter of surveyors entering incorrect data into the survey database. Many years ago data processing errors could occur by incorrectly producing those IBM keypunch cards. Now, data entry is often the result of keying incorrect data in to a computer database or incorrect programming of automated data capture systems.  A good way to avoid some data entry errors is by utilizing computer programs that can check logic consistency across answers to survey questions.

Finally, respondent error occurs when respondents provide misleading information. This may happen intentionally or unintentionally. Respondents may not want to admit to certain behaviors or opinions. A misunderstanding of a question may also lead to respondents giving an incorrect answer. Researchers need to be aware that there are both sampling errors and non-sampling errors.

Reporting sampling errors is fairly straight forward and easily quantifiable. However, non-sampling errors, both response and non-response, are also important to understand. Every effort should be made before actual survey data collection and entry in order to minimize their occurrence.

Effective email marketing is both a science and an art. Often, we focus on creating compelling content with strong calls-to-action. However, we should be thinking beyond just the email. We should be thinking about our entire online marketing strategy. Does your company have a Twitter account, Facebook, LinkedIn, Flicker, YouTube channel? Do you let your subscribers know about them?

Your email templates should include links to your social media pages.

The same works the other way around. The people following you on Twitter are not the same as those receiving your emails. Did you just announce new features to your email subscribers? How will Twitter followers or Facebook fans find out about it? When you send out an email, consider tweeting or updating your Facebook status to spread the news. Not only will this extend your reach, it could help grow your email list.

When you're emailing survey invitations, the same idea applies. Include links to engage with your organization on social networking platforms. If you're looking to increase survey response rates, think about posting the link to your online survey website.

Only 4% of internet users have redeemed a mobile coupon, compared to 85% who have clipped paper coupons and 65% who have printed coupons from the internet or email. This is according to a Honeywell retail survey conducted by Harris Interactive in December 2009. The survey found that overall coupon usage was up in 2009, but mobile coupons are still in the early stages of adoption.

The consumer survey results point to mobile coupon redemption being poised to explode over the next few years, but right now only 10% of survey respondents said they feel comfortable storing coupons on their mobile phones versus printing them out. Half of the 10% wants retailers to text them deals and sales.

Not surprisingly, younger cohorts are more interested in being able to receive and redeem coupons from their mobile phone. Almost half of those surveyed said they were willing to share their cellphone numbers with retailers to receive coupons.

While this eMarketer article implies the lack of mobile coupon redemption is because consumers aren't ready for it, I'm not sure retailers are ready for them either. Personally, I love when I can redeem coupons with my phone, either by showing the cashier my email or relaying a code. It's nice to know I don't have to worry about checking my email before going shopping and printing out the coupons. Instead, I know I can simply check my email while shopping and redeem the coupon right there in the store, no extra paper. Nothing is more frustrating than when you go shopping and get home to find out you had a coupon for the items you purchased that day!

I think market research attitude surveys like this study are helpful to move us all forward. However, it should make you question, What comes first: the chicken or the egg? People aren't going to be comfortable with something that is brand new that only some retailers offer. Until it becomes common practice, I would expect those who are not young shoppers with higher incomes to be wary of changing their shopping habits and attitudes. So the question shouldn't be around how many people are redeeming coupons using their mobile devices, the question should be around how an organization you can make your customers feel more comfortable with the new offering. What benefits does it offer? Show them how easy it is. Let them know what they need to do.

Previously, we had discussed errors in surveying. The two main types of surveying errors are sampling error and non-sampling error. The sampling errors arise from sampling from a list of potential respondents that is different from the universe of interest. Today, we will discuss non-sampling errors. There are basically two types of non-sampling error: Non-response error and Response error.

Non-response error is pretty straight forward but one of the bigger challenges facing researchers today. Unfortunately, the challenge is growing all or the time. Some of the more common reasons for people to not respond to a survey include disinterest in the topic, lack of time to participate, lack or reachability, a feeling of neutrality toward a topic or a misunderstanding of the survey objective. There are other reasons as well, but they all result in the inability of the researcher to get participation from potential respondents.

Over recent years there have been a growing number of laws and regulations that prevent market researchers from contacting people for the purpose of surveying. (This is particularly true in the Asian countries.) Technology has also made it increasingly difficult to reach people. For example, caller ID as part of a telephone service can allow people to not answer telephone calls from parties that they do not recognize, e.g., survey research companies.

If a potential respondent can be reached, the surveyor must provide sufficient motivation to achieve participation. This motivation is sometimes provided by offering an incentive such as compensation or a chance at a prize. Such motivations carry the risk of getting participation from people who want the compensation so much that they will provide faulty (and expedient) information. When this occurs the researcher has overcome “non-response error” at the cost of obtaining response error. There are several types of response errors. They will be discuss in a future article.

Voice of the Customer (VOC) is one of the fastest growing areas of market research. Voice of customer data is a powerful source of business intelligence for your marketing team, regardless of the size of your organization. Sometimes new initiatives can be overwhelming to know how to start, what to do, etc. Thankfully, voice of customer research doesn't need to be a complicated project. My suggestion is to take the leap into customer market research by conducting a website survey.

Before you get started, you'll need to select an online customer feedback software tool. Here are 5 things to keep in mind when shopping for a web based customer survey tool.

1. Budget. You need to decide on your budget for the voice of customer research initiative. There are a range of survey tools out there to choose from. However, not all of them will meet the requirements you have for a customer feedback tool, and you should consider that. If you go with a cheap, or free, solution, you're going to be doing a lot of heavy lifting when it comes to extracting customer insights from your customer data. It's going to be worth your time and budget to look at more powerful, enterprise feedback management solutions.

2. Design and Methodology. Your customer feedback tool needs to give you the flexibility to answer all your business questions. This means having a variety of question types (rating scales, single-select, constant sum, multi-select, matrix questions, etc.) and sophisticated survey question logic. It's a much better idea to vary your question types and keep the customer survey form interactive and engaging than to ask 25 almost identical likert survey scale questions. By having the flexibility in your feedback management software to vary the question types and use question logic to show different questions to different respondent segments, you'll have much more reliable data (and a higher survey response rate)!

3. Metrics. You need to decide on what metric you're going to look at for your VOC program. There are various schools of thought on the best metric. You'll need to weigh the pros and cons and pick the one that will work best for your program goals. One school of thought is to measure customer loyalty with the Net Promoter Score (NPS). Other people believe in the American Customer Satisfaction Index (ACSI). ACSI is recognized as being the gold standard for measuring customer satisfaction, particularly in the eCommerce industry. You can of course choose to go a different route as these aren't the only two options.

4. Reporting. Here's another reason that you should look seriously at professional survey tools instead of free ones. Reporting is essential to customer survey programs (well really any survey program from employee surveys to product surveys to market surveys). Your survey tool should give you the ability to look at dashboards, drill-down and filter by segments. Enterprise survey tools will allow you to select key segments and response criteria, creating charts and graphs automatically from your selections. Be sure your survey software lets you "get your hands dirty" if you want to export answers into Excel or SPSS data analysis tools. Here having the ability to pre-set export values can save you a lot of recoding time.

5. Survey Professional Services. Often times those people who are conducting surveys online or by paper don't have the background for deep statistical analysis. For example, your marketing team, who typically know that most about your business, usually have communication backgrounds not statistical backgrounds. Your survey solution should be able to help you get into the nitty gritty with means, statistical significant, chi-square tests, etc. Think of your enterprise feedback management solution vendor as an additional resource to your current employee base. Web survey professional services not only can provide you additional statistical survey analysis you can't do on your own, but they'll save you alot of time.

When evaluating different customer satisfaction survey software, keep these five points in mind, in addition to comparing features you think are essential for your own survey initiatives. If your survey provider meets all of these points, you still need to make sure they have features like automated email survey reminders or self-cleansing database functions if those are essential to your projects as well.

There are a number of ways that error can enter into a survey. Generally these can be divided into sampling error and non-sampling error. Sampling error occurs when the survey sample is somehow different from the population of interest. Non-sampling error can occur when there are flaws in the survey instrument, e.g., web questionnaire. Both types of error can lead to unwanted and misleading results.

“Sampling error” occurs when a researcher gathers information from respondents who are in some ways different from the true population of interest. For example, a survey researcher may be interested in gathering data on all people that use his company’s product – let’s say automobiles. The surveyor may put together a list from purchase records to identify those who have purchased a car from his company in the past. However, the researcher may not have considered those who purchased his cars on the used car market. Or he may not have considered those who have borrowed or rented his cars. Certainly those people might also be potential customers and it would be worthwhile to ask for their feedback.

Therefore, one must consider in the broadest of terms the universe of potential respondents from which a sample should be drawn. Thinking broadly about your target population will help ensure you attain a representative sample. It may not always be easy to reach the correct market research survey sample frame but not doing so can lead to missing information and missed opportunities.

It's helpful to define market research terms to better understand the whole market research process. A later article will discuss the ramifications of “non-sampling error”.

Every good market researcher knows how to turn raw data into a comprehensive and informative report. But some market research reports require thousands of survey responses, and this can be a daunting task. How can you maintain a handle on such a large project, while keeping the data accurate and organized?


Please share your tips for project management below – I’d love to hear them!

For certain surveys and certain groups, a direct mail questionnaire is a good option.  The best time to use this method of gathering data is when visual aids are required, and lengthy open-ended response are not.

For example, if you are conducting a survey for a cereal company about their packaging, you can send a sample package and the direct mail survey to a respondent pool. They can answer the survey at their own convenience.

How can you ensure that recipients will fill out the survey? There are several things you can do to guarantee a completed response:

Price almost always is a major concern for clients engaging in a market research project. Often the next biggest concern, albeit a close second, is time (when can we have the results?). A third aspect which can enter into the project discussion is quality of the results. Some clients prefer to assume that the quality of survey results is a given and do not pay close attention to the attribute. Making that assumption can be a serious mistake.

Researchers and clients should be aware of the interdependence among price, time and quality. Realistically, clients can choose two of the three. The third variable will be a default. Purchasers of research may hope to dictate that they want the highest quality data, on the cheap and they want it now.  But a good understanding of expectations vs. reality is important for both happy clients and vendors. Stated another way, one can expect to get high quality data quickly if they are willing to pay enough for it. Also they can expect to get results fast and cheap if they are less fussy about quality. And finally they might be able to obtain high quality results for a bargain price if they are willing to wait longer for the results. But good, cheap and fast is a hard order to fill.

Online surveys have made great inroads in to satisfying high expectations of research clients. The internet has opened excellent opportunities to reach appropriate target audiences, quickly and at a reasonable cost. One must still be careful to understand the inherent bias in online surveying. Using quotas are a good start. Panels are another good method of knowing that you will be connecting with customers and potential customers. But all in all, online surveying has been a boon to both researchers and their clients.

More and more market research client’s may be interested in piggybacking, or combining funds to complete a research study. This is beneficial for all parties because you might gain new clients who would not otherwise be able to afford your services, and they will be happy you can accommodate them.

Piggybacking lends itself more easily to certain types of studies. Employee surveys, customer satisfaction questionnaires or product-specific studies may not necessarily be the best fits, but general state of the marketplace studies are perfect.

Work with all the parties to collaborate and write a questionnaire that encompasses all their needs. If sending an online survey, be sure that the survey sample meets all parties' requirements. Find out what type of data each one needs and write the survey to ensure that you can answer all their questions.

You may have to make separate market research report presentations of the data for each organization, to fit their criteria. You can also make a larger, comprehensive presentation or whitepaper for all of them. If they are interested, you can even do a press release about the whitepaper listing them as sponsors of the study. This could help establish them as leaders in their field and inspire more clients to contact you about joint studies.

Last week, we looked at one common problem with marketing surveys and how it can be solved through sending your surveys from email addresses or websites that are recognizable to your audience. 

Lets now take a look at more ways to ensure your customers receive good survey market service every time. The answer to good market survey service may seem obvious. Yet, few, if any, market researchers utilize any of the suggestions below; all of which have proven to be effective in increasing participation in market surveys.

Good market survey service can include:


Of the 4-5 surveys I talked about participating in in the past few months, only one of the companies sponsoring the feedback survey form took advantage of all 4 points towards good market survey service.

Collecting the right information to make a rational and informed marketing decision sometimes simply means using your knowledge to make a decision on the spot. At other times, it entails collecting an enormous amount of information, or simply put data: the facts and figures related to the problem. Facts and figures that have already been recorded before the project at hand are Secondary data, whereas Primary data are facts and figures that are newly collected for the project.

Gathering primary data during marketing research is only the first step in retrieving the necessary information to continue in the marketing research process. Primary data can be supported significantly with the use of secondary data. Secondary data can greatly substantiate all information gathered during primary research if primary research is even required. In some instances, secondary data alone can solve the problem, eliminating the need for primary data.

There are many ways in which a marketing researcher can retrieve secondary data. The largest and frequently least expensive source for secondary data is the World Wide Web. There are many reasons as to why secondary data is utilized, especially from the internet. The first and foremost reason for the use of online secondary data for marketing research is because almost any data imaginable is readily available. Secondary data can be retrieved from thousands of places on the internet. Many organizations list secondary data on the internet for instant access to marketing researchers. This allows marketing researchers quick access to information, which consequently, allows marketing researchers to make faster, more precise decisions.

Not all problems can be solved with the use of secondary online data. Despite that online secondary data is low cost, by utilizing secondary data over primary data, companies can save thousands of dollars in marketing research expenses every year. A focus group or depth interview is always more appropriate than online secondary data when a corporation needs to know the inner most thoughts and motives of customers. Information retrieved through internet market research tools always calls into question the legitimacy of the research findings because of the ever growing presence of false and inaccurate information.

One of the more used and abused aspects of market research is the conclusions that are drawn form a project. A research project can be well designed and efficient, and yet when it comes to interpretation and (hopefully) implementation, the stakeholders can make incorrect decisions simply because they failed to understand and accept one of the basic tenets of statistical analysis. The oft forgotten concept is the difference between correlation and causation.

We see this every day. A new study comes out linking some event with another that may or may not be associated in any way. But the results are reported to imply that one thing is causing the other to happen. In actuality, a researcher has determined a correlation between two or more variables. Those reporting the findings then state the results to imply that one event in some way caused another. Correlation is NOT causation. Of course to explain the science of market research and what statistical results really show will likely bore the vast majority of the target audience. So we hear the one or two sentence thought byte on the radio or TV and many are fooled in to thinking there is something important happening. In fact, like an ice berg, we’ve only seen the tip and the tip can be so misleading.

Here is a (somewhat fictitious) example:

I’m all for using market research in main stream enlightenment, but we, as research professionals, must always take a critical eye to what we see reported in the media. When there is no relationship between two variables but the analyst concludes that a relationship exists - know as a "spurious correlation" - we need to be aware and make others aware of that possibility.

The number of simple online survey tools has enabled even the smallest organization to conduct an in-depth market research analysis for a fraction of the cost and time. But these feedback collection opportunities can still be wasted if you don’t determine proper survey objectives before you begin writing survey questionnaires. Without defining your marketing research, employee feedback form or customer satisfaction survey objectives before hand, your questions may turn out to be irrelevant, unnecessary or even misleading. Strong, defined objective give you clarity when designing questionnaires.

Suppose you are doing a survey looking for information on pricing, packaging and product satisfaction for their laundry detergent. You want to narrow in on these topics to create a short, 15-minute online questionnaire. Don’t overload respondents with questions that don’t matter. Test out the web survey yourself and scrutinize the answer options, can you come up with action items based on the survey responses? If you find that the data analysis reveals there isn't enough usable information, see what questions can be rewritten or replaced to get you the data you need. If necessary, split the survey into parts to illuminate each item you need information on.

The same criteria should be applied to internal surveys, like employee opinion surveys. Determine key areas, like pay, benefits and work environment, that you want to collect employee feedback on and write the survey to highlight those areas. With an internal audience like employees, you have the luxury of surveying them several times a year if necessary, but you should still plan the objectives carefully.

Market research and marketing research are misleadingly-similar terms that describe distinctly separate concepts. "Market" research is, very simply, research into a specific market. It is a very narrow concept. "Marketing" research, on the other hand, are much broader. Narrow-scope “market” research is but one specific element of a comprehensive “marketing” research process, which also entails research into areas such as new products and modes of distribution (via the Internet, for example). According to the American Marketing Association, marketing research is the systematic gathering, recording, and analyzing of data about problems relating to the marketing of goods and services.

To be effective, marketing research activities must be thoroughly planned and systematically executed. Process-critical steps include:

1. Problem definition. Never conduct research for things that you would like to know. Make sure that you really need to know something. The problem then becomes the focus of the research. For example, why are sales falling in the Midwest market?

2. Data collection method. How will you collect data that you will analyze to solve your problem? Do you conduct a telephone or an online survey; arrange a focus group or one-on-one interviews?

3. Sampling method. Do you use a random or stratified sample? In a random sample, you would select subjects randomly from a single large pool. In a stratified random sample, you would divide this large pool of subjects into several groups (strata) and then randomly select subjects from within each group.

4. Data analysis. How will you analyze any data collected? What software will you use? What degree of accuracy is required?

5. Budget and Timeframe. Decide upon what resources you can allocate for this project. Go back to the managers or clients requesting the research. Make sure that you agree on the problem definition, the budget and timeframe. 

6. Data collection. Go ahead and collect the data. Basic source of data could be either secondary or primary. Secondary data is data that has been collected by someone else for other purposes. Primary data is the data that you collect yourself from scratch.

7. Data analysis. Conduct the analysis of the data. Types of statistical survey data analysis that might be performed are simple frequency distributions, crosstab analysis, multiple regression, cluster analysis, factor analysis, perceptual mapping, structural equation modeling and data mining.

8. Check for errors. It is not uncommon to find errors in sampling, data collection method, or analytic mistakes.

9. Reporting and presentation. Write the final report. It may contain charts, tables and diagrams that will communicate the results of the research, and hopefully lead to a solution to your problem. Watch out for errors in interpretation. Any critical information and knowledge that comes from your market research investment will be limited by how your market research reports are presented to decision makers.

Perhaps the most important aspect of a truly value-added marketing research process is EXECUTION. All of the data and analysis in the world is for naught if the organization fails to act upon information. The only thing worse than not doing market research at all is spending money on it and not utilizing the results.