Online Marketing Research

When you plan your online market research study, you still need to remember the standard steps for any market research project, plus some online specific tasks:


The time frame for an online study is, as discussed previously, often shorter because of the immediacy of the results. Keep this in mind but be sure to give yourself enough time to analyze survey responses.

Increasing survey response rates is a major goal of most survey builders and market researchers. There's an art and a science to increasing campaign response rates whether it's an email marketing campaign or an online market research study. I wanted to share some of my tips for how to increase survey response rates:

• Make the email survey invitation from names easy to recognize. You can do this by including an individual's name within the organization that's well known (such as the CEO or if it's a client survey, the name of their sales rep). You can also use the organization's name, or both. For example, I'm subscribed to a few MarketingProf's newsletters. When they send out emails they include the same person's name and their organizations name so it looks like this: Anne, MarketingProfs. I recognize it everytime, and since I enjoy their newsletter, I made sure to open the email.

• Keep subject line's compelling, but short. The subject line and the From Name are the two most critical pieces to get your email opened. Try to keep your subject lines to 35 characters or less. You don't need to put the entire email in the subject line, but you do need to include enough information to make the recipients open the survey email.

• Create an attractive survey invitation. Studies have shown that well done HTML email messages get better response rates than plain text emails. With HTML you have the opportunity to include images, change font sizes, bold text, etc. Take advantage of this chance. It's one more way to get people to click through to your survey!

• Send personalized survey invitations. Personalizing your emails, even something as simple as including the recipient's name in the greeting, will return a higher response than a generic message. It creates a personal touch, and makes the recipient feel like someone took the time to send them a personal message (even though your email survey tool did it for you).

• Introduce the survey. Let the participants know why they should participate in your survey. If they don't understand why their opinion is important to your survey findings, why would they want to take the time to fill it out?

• How long will the survey take? Not setting expectations in the beginning for survey length leads to low response rates and high abandonment rates. Not what you want to see. If you don't let people know how long it will take to fill out your online survey, they're going to assume you're hiding something about how long it is. Tell them it will take X minutes or the survey is only Y questions long. Definitely be honest, if you lie here, you're going to hurt your future chances of getting those respondents to complete your next online survey questionnaire.

• Remind your survey sample that their responses will be kept confidential. This is particularly important for surveys about uncomfortable topics. For example, you created a poll for a public opinion survey to see how your population feels about an emotional topic such as abortion. If you don't keep the information confidential, you probably will not get honest feedback. Same thing goes with employee questionnaires, they should always be kept confidential and anonymous. Not keeping responses confidential will definitely hurt your response rate, as well as the validity of the data.

• Offer an incentive. Offering incentives is a proven method for increasing survey response rates. But this method doesn't work if you don't let people know about it up front. Put it in your email, put it on the welcome page of the survey, then make sure to follow up. Again, if you drop the ball here, the chances of that survey respondent completing your online survey form in the future is drastically reduced.

• Always say Thank You! Remember when your mom always made you write thank you notes when you got presents or cards from people for holidays and birthdays? It was because people like to feel their effort is noticed and appreciated. Same idea here, it's nice to just get a short note thanking them for their time. After all, survey respondents are doing you a favor.

• Don't over email your contact list. This is very important. You shouldn't be inviting the same people to complete your online surveys every month. It's important to segment your list using whichever survey sampling method that works best for your surveys to avoid email list fatigue. Make sure that you're coordinating with other campaigns as well. Just because you know they're different initiatives you're emailing about, your contact's wont necessarily see it that way. If you begin sending emails too frequently, contacts are going to just delete your email and never open it, opt out or report you as a spammer.

Have a tip that I missed? I'd love to hear it!

As is the case with validity, there are multiple aspects of reliability in survey research.  In general though, while validity refers to accuracy in question design, reliability refers to the consistency of your results (i.e. the probability of producing the same results after repeated measurements). 

One way to easily conceptualize reliability is to consider it as another word for precision.  Let’s use archery as an example: if you shoot ten arrows at the target and they all hit the same area, your shots can be said to be reliable or precise. 

You can test for and estimate the reliability of your survey questions in a variety of ways.  You could use a test-retest design, where you use the same questionnaire more than once to see if the results remain consistent.  You could also send your feedback form to different survey samples (consisting of similar respondents) and then compare your results for any fluctuations.  Also, if you create a set of customer service questions designed to provide you with a composite scale, you would want the individual questions to produce similar results if they are intended to measure a single concept (an overall customer service scale, for example). 

It is also important to understand the relationship between reliability and validity; results can be reliable without being valid.  If we use our archery example, the shots are reliable, but if they aren’t near the bullseye (an undesirable outcome!) they cannot be said to be valid.  Reliability is necessary for validity, but it is not sufficient alone.  You want to accomplish both objectives: shots that land near the bullseye consistently.  By creating reliable, valid questions for attitude surveys, online market research, employee reviews or public opinion polls, you can be confident that your results are not due to chance.

A new public opinion poll was commissioned by SCHOTT Solar and Solar Energy Industries Association (SEIA) for the annual SCHOTT Solar Barometer Survey. The attitude survey found 92% of Americans think developing and using solar energy is important for the United States. With all the political debates going on in Washington, this does not rank among them. Survey responses were consistent across all political affiliations and ideologies. 77% of survey respondents indicated they think the development of renewable energy sources should be a major priority for the federal government, this includes providing financial support when needed. The most favored renewable energy source is solar (44%), followed by wind (17%), natural gas (12%) and nuclear (10%).

The private sector sees the possibilities in the solar space as well. They aren't waiting for a web survey to tell them. According to Cleantech Group, the "cleantech sector" has grown almost 25% over the last 5 years. Just in the last quarter, solar was the leading sector. There is still plenty of room for growth in this area though. The SCHOTT survey found that almost half (49%) of the survey sample are currently thinking about solar power options for their home or business. This is a big opportunity for organizations looking to develop solar energy solutions.

One think I always tell you to look for is the survey methodology used to collect the data and draw conclusions. According to the SEIA press release, a third party market research firm conducted the online survey between August 31 - September 8, 2009. Email survey invitations were used to solicit survey responses to the online market research survey. Due to the quantitative market research method, the survey results are 95% accurate within plus or minus 3.1 percentage points.

According to new survey data from the Advertising Research Foundation (ARF), online survey findings are just as accurate as results from a telephone survey.

ARF conducted survey research using three different survey methods to reach this conclusion: 1,500 paper questionnaires, 1,000 telephone surveys using random-digit dialing and 100,000 web surveys using 17 different panel providers. However, the telephone survey sample did have it's biggest failure in the age of the respondents because of the increasing number of "cell-only" households.

When you're deciding between different survey techniques and research methods, don't let the common myth that telephone surveys yield more accurate survey results sway you from using online market research methods - such as conducting online survey research.

Joel Rubinson shared some of the survey data collected on his blog. You can read what he has to say here.

It keeps coming up in the social media space when talking about market research data:


It depends on the target market you need for your marketing analysis. Twitter, and other social networks, are only going to represent part of your target market. For example, your early adopters may be on Twitter but the rest of your customers are not. If early adopters are the people you want to collect feedback from using an online market research survey software, Twitter is a good place to start asking product research survey questions. On the other side, Twitter may not be the ideal place for conducting an online survey targeting customers who use a wheelchair.

The danger with using social media for data collection is adding bias into you market research process. Online survey tools can certainly help you avoid various types of survey biases, but you do still need to think through your survey methods and process no matter what marketing survey software you choose to create marketing surveys.

When I think about Cvent Web Surveys software features I typically think of our robust contact database that allows you to track activity history and segment lists; our email marketing engine that allows you to send survey invitations; our libraries that allow you to quickly select pre-designed graphical templates, utilize pre-built survey question and responses to get their survey out the door quickly; and survey question logic ranging from sub-questions to skip and branch to pipe and finally other advanced logic.

However, I often overlook the fact that clients can customize the footer of their online survey questionnaires. It should be no surprise that we empower our users to use the footer associated with the graphical survey's template, create their own footer image and add footer text. A survey should reflect your brand, from the header to the footer and throughout the graphical layout of the survey. Some survey companies force survey designers to have their default footer on the bottom of every survey. Instead, we give survey designers complete freedom over the online survey's design.

So what might you even want to put in your customized footer? I would tell you it depends on the type of survey you're doing. If you're offering an incentive to generate survey responses, you may want to include a link to any restrictions or contest rules. If you're collecting personal information in the survey, it may be important to include your organization's privacy policy. If you're conducting a branded survey (versus a blind survey), company logo and other information may go in the footer of online market research or customer surveys. Want your survey footer to exactly match your website's footer? Go for it. Being able to match your survey templates to your website is one of the reasons to chose our web survey solution.

Ready to get started building your own online survey with a customized footer? Sign up for our free trial account and get started today.

MR Heretic recently posted about Phishy Survey URLs. The main point is how having a weird looking URL for your online survey could lower your response rate and destroy the customer trust your organization has worked so hard to gain. Why? If your organization is including IP addresses in your web survey link, consumers will think it looks like phishing and decide against clicking the link. By not clicking the link in your email marketing survey invitation, customers are opting out of your online market research or customer survey.

My recommendation: use a customer survey software or email survey tool that allows you to brand your own survey URL without any IT staff or programming knowledge. Cvent's Web Survey Software gives each survey it's own unique URL automatically, but you can take that a step forward and include your organization's name in the URL as well.

Getting clients to respond to your survey is critical for any survey project's success. There are enough obstacles standing between you and the desired completed online survey submission - don't ruin it by using a phishy looking URL.

Social media seems to be taking over the world of marketing. A lot of discussion has focused on how to set up a Twitter account, and how to use it to listen and engage with customers. Many people are beginning to call social media the new medium for market research and surveys. But I would have to disagree. While it's an excellent idea to listen to what people are saying about their needs, your industry or your products, other qualitative and quantitative research methods are still essential. Social media shouldn't replace online communities, focus groups, etc. They still have their place in the rapidly changing world of online market research.

What really keeps me up at night is the idea of people distributing their surveys through social media. For some surveys, this could be an okay approach. The problem with it is the data. If you just get responses through these channels, there's no way your survey results will be representative of your target market, unless your target market is your Twitter followers. If your target population happens to be your Twitter followers, then your survey will be representative because you're inviting everyone (a.k.a. conducting a census survey), but how likely is it that you only want the opinions from that small group?

If you do decide that a non-probability sample is a requirement for your online survey project, try to find additional channels to solicit feedback responses through in addition to social media.

I cannot begin to explain how frustrating I find telephone surveys. They drive me crazy. I'm always happy to share my opinions with organizations but don't waste an hour of my day on a bad telephone survey. A bad survey experience reflects on the brand. Respondent's experiences converge into the customer experience, they don't keep the buying and the surveying experiences separate in their minds.

It's easy to create a questionnaire online with web survey software and finding online survey tools that fits your needs is easy. So why are there not more online market research studies being done? Why does my phone ring every night, while I'm eating dinner, with another request to complete a telephone survey? Technology improves the respondent experience tenfold. As I hinted at above, respondent experience impacts customer experience, which influences customer loyalty and retention. If you think you need additional channels for your data collection other than online, here are a few telephone survey tips to help the respondent experience:

1. Training is essential. Make sure your interviewer is well trained and understands their job. For open ended questions, have the interviewer read back the response, ask for more details, and clarify the response they have recorded is exactly what the respondent said. Interviewer bias has a tendency to creep into open ended questions and training is the best way to combat bias.

2. Vary the question type. Asking more than a few of the same question type in a row becomes tedious for the respondent. Respondents also lose interest and you begin to lose their focus. Alter your question types to keep respondents engaged and giving honest feedback.

3. Speak English. With more and more organizations and market research agencies offshoring their telephone surveys, it's really important to ensure your interviewers speak clear English. It should not, at any point, become the respondent's responsibility to try to figure out what the interviewer is asking. Remember, you called them and asked them to help you. In most cases, the respondent isn't getting anything out of it.

4. Only ask important questions. This is a best practice for all survey types whether its a web based survey or a paper feedback form. If you don't need the answer to the question, take it out. If the question does not directly relate back to the survey project objectives, take it out. Do not ask the same question three different ways, one way is good enough. Respondents completing an online questionnaire may be more forgiving of extra questions than in a telephone survey, don't waste their time with extra questions.

The only time I recommend using telephone surveys is when you're trying to build a relationship. Asking for customer service feedback or conducting customer satisfaction surveys are excellent opportunities to take the time to call your clients. The personal touch can go a long way to improve customer retention programs and consumer loyalty.

The person who coined the phrase "Looks don’t matter" probably never tried their hand at surveying. We’ve stated at least a few times in the past that the visual appeal factor has a definite impact on survey response rates. However, there are other very compelling benefits to making your surveys look fabulous.

1. Visual Appeal Reinforces Consumer Awareness

A survey is, in a sense, a marketing piece. If you’re conducting market research on an unreleased product, you want to make sure your respondents keep a positive image of that product in their memory. After all, they may potentially have a need to purchase from you someday in the future. A good-looking marketing piece is a great way to create initial awareness.

Even if this is a post-purchase or customer service survey, a professional-looking design can have a visual impact that reinforces the product’s positive and trustworthy appearance.

2. Visual Appeal Encourages Honest Feedback

Especially when surveying a population of leads, prospects or other online market research audiences, it is important to achieve a level of visual appeal such that it extends your company’s professional brand image. First of all, you do not have a solid business relationship with this audience yet, so it is very important that you earn their trust. With these types of surveys, you may ask for personal information such as phone numbers and addresses; the last thing anyone wants to do is mistakenly sign up on a page and compromise their personal data. Having your survey pages reflect your company site helps combat this problem and encourages respondents not only to complete your survey, but to also give honest and complete feedback.

A good way to do this is to have your survey match the look and feel of your company website. In fact, it’s advisable to customize your headers, footers and other survey sections to match everything from color and style of text to the page layout. A good way to do this is to get the appropriate sections from your own website and drop the code and CSS styles into your own brand survey.

Below is some work we did for a client last year. We examined our client’s web page and then created a survey design that stayed consistent with the corporate home page:

Company Website:



Online Survey Welcome Page:



We were able to match the look and feel of the company’s corporate page by utilizing elements of CSS to match the colors and fonts, then added the same flash animation on the header to really jazz it up.

There's nothing wrong with using graphical templates in surveys, but having the option to completely customize your survey pages and emails helps you get to that next level of survey effectiveness.

One area that many of our clients have questions about is reporting. As in, How do I make meaning out of all this survey data? What types of reports offer the appropriate insight in this case?

It’s no surprise that our relationship managers spend the majority of their day helping clients figure out ways to create meaningful reports. Understandably so, considering that the information contained within these reports is likely the initial motivation for their decision to conduct a survey.

There is no one right way to go about analyzing survey results, but there are a few best practices on how to analyze survey data that can really simplify the entire reporting process.

1. Work Backwards—Plan the Report Before Planning the Questions

We often tell clients to work backwards when planning their survey program. Rather than jumping into question creation, a survey planner should first have an idea of what the final report will offer in terms of useful information. If you are conducting a market demographics research study, and you are interested in knowing what age group is most receptive to an ad message, it is important that your survey ask age (or age range), rather than their date of birth. In the latter case, the same data can be extrapolated, but it creates extra work for you when it comes time for reporting.

Also, if you plan to include statistical information in your report (mean age, mean income, etc.), you want to make sure that your survey questions will elicit numerical responses. Don’t ask a respondent for their annual income with a one line comment box—your data will become very messy with responses like, "$35953.00" and "60000 USD." Again, know what data you want, and what you want to do with it first. Then, ask the questions to fetch this data according to your plans.

2. Use Filters to Drill Down—Segment your Data to Reveal Insight

Segmenting your respondents into targeted groups is a best practice we advocate often. It is just as important, however, to also segment the data when running analyses in order to reveal a deeper level of insight. For instance, your survey results may state that the average satisfaction rating from your entire client base is 4.7 out of 5. Upon further research, however, you might find that the average rating in most states is 4.9 or 5.0, except for the Northwest states, which gave an average rating of 4.1—clearly under company standards.

Without using filters to segment your data, you can overlook vital information that can affect your business. Filters can be used to separate and compare groups of data based on virtually any data point you have already in your database, or those that you might collect in the same survey.

3. Every Good Report Deserves Another—Observe Trends Over Time

No survey is an island. It might sound like extra work, but if you are looking to achieve real business results through your survey initiative, then your plan needs to include conducting the same survey again at a later time. Only by doing so can you observe trends and keep your organization accountable for the metrics you are aiming to manage.

If you are surveying your customers about telephone technical support, you want to ensure that you can pull cross-survey reports that show any progress or decline in feedback ratings. Such information is invaluable in directly managing a team; when such a survey is integrated into a team’s incentive plan, for example, these metrics can be instrumental in motivating employees to work harder and better.

Ongoing surveys that can be analyzed against each other with these cross-survey reports help you track changing opinions and needs so your organization can react efficiently to unexpected downturns, or perhaps increase revenue through new opportunities. Survey data, unfortunately, does have an expiration date, and in a dynamic business world, it is necessary to conduct ongoing surveys and accordingly adjust.

Imagine the pool of data that can come back from a market research survey:

• 85% Male, 15% Female
• 70% use Windows, 22% use Mac OS, 8% use Other
• 12% C-Level Executives, 39% Managers, 49% Non-Managers
• 8% Highly Interested, 29% Interested, 42% Somewhat Interested, 21% Not Interested

Without connections, or associations, between these data points, this market research can lead to poor decision making. At first glance, a software company executive might think, “We should cater our product toward a male audience with appropriate advertising. The CD that comes with the product should be for Windows. After all, look at all the decision makers (51%) and those showing interest (79%)!”

Again, at first glance, this may seem viable, but without tabulating the correlation values between each of the data points, it is impossible to make such assumptions! What if the 15% of females surveyed constituted all the C-Level Executives and many of the decision making managers, while those who stated a high interest were mostly Mac users?

The only way to identify such connections is to use cross-tabulation analysis with your survey data, which can reveal valuable associations between any two data points of your choosing. Commercial grade survey technology should have built-in cross tabulation capability so you can avoid doing complicated statistical calculations in multiple applications.

Make sure you can quickly and efficiently reveal these connections before relying on your market research to make decisions.