Survey Examples

I realized it only after the emails had been sent.  It was a test survey that wasn’t intended for entry level employees and senior management.  It was an employee evaluation that had initially been slated to go out only to our newest batch of hires.  Only last minute did we discover we could use logic to create one interactive survey online for senior management to evaluate the progress of new hire training, and new hire survey questions for them to evaluate the training process.  The wording, the slang, everything was wrong!  What did I do…

Hopefully, this scenario has appeared solely in a nightmare (and yes, survey nightmares are very real and very terrifying).  But a similar, less daunting situation is common among many: You have an online web survey that will go out to two groups of respondents, the wording in the two would ideally be completely different, and you do not have the time to go into the two templates you’ve created and select the different audiences then manually send the emails.  Not to mention, this method would require using Cvent’s email survey tool's default templates and the Custom Messages.  What if I told you it was possible to create entirely separate email marketing campaigns, set the emails up to go out automatically AND keep the both default templates and custom messages for each group of respondents? 

The multiple email marketing campaigns feature allows you to do just this!  You can select the number of separate email blasts you intend on sending, you can add entirely different contact groups to each campaign, and you can set up your emails to go out automatically. 

1. This feature is perfect if you have a multilingual survey.  You can translate your message into as many languages as you like to ensure maximum comprehension. 

2. Multiple email campaigns is also great for newsletters that are tied to a generic instant survey.  Instead of copying your survey and creating a new email blast every month, you can add an email campaign for the October '09 Newsletter, November '09 Newsletter, December '09 Newsletter and so forth.  You can then get an aggregate view of the data collected across all months.

3. The nightmare scenario outlined above.  It is likely that emails sent to your senior management are not worded the same as emails sent to entry-level new hires.  The formality of the verbiage aside, your new hires might be a bit disconcerted to know that the bosses of the bosses are watching them and evaluating their decisions. 

We all know creating surveys and designing questionnaires can be a bit overwhelming.  Why put yourself through that again just so different people can receive the same questionnaire?

I’ve talked extensively in the past about using Facebook for market research. But there are other Web 2.0 sites that you can use effectively as well.

For example, Twitter is growing more and more. You can search on Twitter for keywords or competing products to see how often they are mentioned, and what people are saying. This is a quick and simple method of gathering background info.

There are many sites where people mark things they like or dislike, like digg and del.icio.us. Everything on digg is submitted by users and then ranked by users. After a story has enough “diggs” it is listed on the front page. You can search for your client and see what types of stories are making their way to the site and get an idea of what kind of buzz is surrounding them. Del.icio.us allows users to tag and bookmark their favorite pages, and then allows all users to access rankings. You can find out how many people have bookmarked your client and competitors.

Of course, both these groups are by their nature narrower than most studies require. Results for each page are primarily involving only regular users of the site. You should broaden your search to find other respondents so your data is more accurate.

Web 2.0 sites are great places to find information and do background research. They also tend to be good places to find electronic survey respondents, as users are web savvy and opinionated. But as I mentioned, be sure not to make your respondent range too narrow.

In yesterday's post we looked at how your survey question items, if worded improperly, may be biased, resulting in unusable or incorrect results. Now we know how survey questions shouldn't look, let's delve deeper into why those previous examples were poorly worded, and how with just a few small adjustments you can easily start to hone question items within your survey forms.

When you design internet surveys, how do you write objective questions that won't bias your results one way or the other?

Here are a few main points:


These first few points are a great starting place when you're designing a questionnaire. The final post in this series will look at a few final survey best practice tips for writing great survey questions and the benefits you'll gain from crafting quality items. In the meantime, questions previously stored in our question library already come properly worded and ready for use!

Vocus, a leading provider of on-demand public relations management software, released the results of a recent public relations and marketing survey they conducted. The survey sample was made up of 1,800 marketing and PR professionals. Among Vocus's key findings was the response of 80% respondents that social media would be a key focus for them in 2010. Vocus shared that they believe social media has been a catalyst for the change in the PR industry, where PR is becoming increasingly more important.

How is the PR industry dealing with the fact they're being asked to do more with less? Investing in technology. Sounds like a familiar story, right? Marketers turn to email survey tools to send out email survey invitations instead of having to manually send all email marketing messages through Outlook. Accounting departments use software to help them manage finances instead of the old way of paper, pencils and ledgers. Technology is often used to help professionals manage their work loads, and PR industry is doing the same thing.

Here are some of the other key findings Vocus shared from it's marketing survey:


You can learn more about the survey results by viewing the recording of webinar Vocus hosted yesterday, PR Planning Considerations for 2010.

Designing a survey instrument can be one of the most fun aspects of conducting survey research. In this two-part tutorial, we'll first look at some of the "Don'ts" of writing survey questions, and then we will look at some of the "Do's".

Once the brainstorming of ideas phase has been accomplished and you have a good idea what the scope of your research is, it's time to get down to putting question ideas on paper (or online!).

However creating good survey questions can turn into quite an art form, as we all struggle to word our items in such a way that our participants will understand what information we are asking for. Recognizing when you've accidentally designed a survey question that is worded in a non-objective way is the sign of a good researcher. Let's see if you can spot the poorly worded question from this group of sample survey questions:


Now that we've looked at some question items that shouldn't be put into surveys, next time we can define some ways we can create objective question items that will provide us with the results we need!

In a recent post, I described the differences between nonprobability and probability sampling methods in online survey designs.  Probability methods are preferred if at all possible, because they allow you to make generalizations from your electronic survey results to a larger population or target audience.  One kind of probability-based sampling technique is called systematic random sampling. 

To employ a systematic random sampling design for your online web survey, you first select a case at random from your exhaustive population list, and then select further cases at identical intervals, determined by how many people you want to sample in total.  If you wanted to sample ten people from a population list of 150, you would then choose every fifteenth person after selecting someone in the first 15 cases (to ensure you will select 10 people in total).

This provides an easy way to obtain a random sample of your population list or sampling frame, because as long as your data is ordered randomly, you can begin simply by selecting any record or case and go from there.  This is an important caveat though: your records must be randomly ordered for a systematic sample to be effective. 

Take this example of survey sampling, let’s say you have a sampling frame (list) of people that is currently ordered alphabetically by last name, and you are interested in subgroup analyses by ethnicity.  It would be wise in this case to rearrange the records into a truly random order (i.e., not alphabetically), because last names from certain backgrounds may be more likely to begin with a particular letter.  While systematic sampling provides an easy way to generate a random sample for online surveys, you do need to be sure there is no hidden order within your population list or sampling frame.

There advantages and disadvantages to conducting web or electronic surveys as opposed to traditional survey modes such as personal interviewing, telephone and mail.  One of the advantages of an online survey design is a possible reduction of what is known as social desirability bias. 

Social desirability bias occurs when survey respondents offer responses that portray them in a positive or more favorable manner to others. 

When a face-to-face interviewer asks personal or sensitive questions respondents feel have a “good answer” and a “bad answer” (such as criminal behavior), they may underreport bad behavior and overreport good behavior, for example.  While this bias may be reduced in online surveys due to the absence of an interviewer, there are topics that may produce invalid or unreliable data, regardless of survey mode.  This is something to consider before finalizing your online questionnaire design or web survey forms.

Survey questions within the following content areas are especially subject to social desirability in a survey form (i.e., respondents believe particular responses are “better” than others). 

A key marker of a quality survey, whether that survey is an online form or another mode, is replicability of your results over trials. Replicability of your survey results lends credibility to your organization's research work.

One indication that your survey form may have results based on biased samples is nonreplicable results for the same instrument, fielded with the same survey design, from one time to the next.  If the same sample frame produces significantly different results for identical questions, that should perk up our noses to the scent of possibly unreliable results. We should start to look at the criteria used to select our survey sample, and take a fresh look at the questionnaire to see if it is still pertinent to our needs, or the needs of our clients.

This holds true for all modes of survey research, including online survey internet research. Receiving divergent findings in survey results being fielded again and again could lead to differing (and possibly harmful) business decisions.  No company should be left with uncertainty about answers and actions when decisions need to be made based upon the research. This is just one reason why replicability of your research findings is a key ingredient to sound data. When in doubt- replicate!

It is crucial that you consider reporting a main element of your web survey design at the outset of your research project.  What you can say about your results hinges heavily on the types of analyses your questions and the capabilities of your response scales.  Today, I will outline the difference between the two major branches of statistical analysis available for most survey data: descriptive and inferential.

Descriptive statistics are the basic measures used to describe survey data.  They consist of summary descriptions of single variables (also called “univariate” analysis) and the associated survey sample.  Examples of descriptive statistics for survey data include frequency and percentage response distributions, measures of central tendency (which include the mean, median and mode), and dispersion measures such as the range and standard deviation, which describe how close the values or responses are to central tendencies.

Inferential statistics offer more powerful analyses to be performed on your online web survey data.  As the names suggests, this branch of statistics is concerned with making larger inferences about social phenomena.  This can include associations between variables, how well your sample represents a larger population, and cause-and-effect relationships.  Some examples of inferential statistics commonly used in survey data analysis are t-tests that compare group averages, analyses of variance, correlation and regression, and advanced techniques such as factor analysis, cluster analysis and multidimensional modeling procedures.

By designing online questionnaires and survey web forms with a good idea of what you want to do with your data after it's collected, you can create cohesive, powerful reports and presentations. Need more tips for how to analyze survey data, read some of these data analysis posts.

What is a survey? The Merriam-Webster Dictionary defines the word "survey" in this way: "To query (someone) in order to collect data for the analysis of some aspect of a group or area."  These "someones" can also be called your survey sample. Unlike a census, which surveys an entire population, a sample survey gathers information from only a fraction of the individuals within an entire group. Samples are collected in such a way that results from the survey can then be generalized back to the entire population.

Surveys can be conducted in a variety of ways, which can also be termed your survey design. They can be conducted over the phone, through responses written on paper, through in-person interviews, or through online web surveys. Even though the mode in which a survey is conducted can vary, sample members are all asked the same questions in a similar or standard way.

Some survey forms focus on opinions and attitudes (e.g. asking your clients how they feel about a new product line), while others focus on factual questions (household income levels, or age of members). Most online survey design contains a combination of both types of questions. A survey design can also vary in length and time it takes the sample to complete. Some may take only a few minutes of a person's time, while others may take hours or even days to complete. 

Surveys should be carried out only for the purposes of gathering data and information on a certain subject. A quality survey can be determined by looking at the way in which it was conducted, and its purpose.

Get started designing your own survey using one of over 50 graphical templates available, or read more blogs on designing surveys.

There are many options available to market researchers when designing a web survey form. I covered this topic a bit last week when I discussed creating online surveys, and included some dos and don’ts.

Conducting a study for the Web offers several opportunities for market researchers that paper questionnaires do not. For example, you can create an instant survey and check on responses frequently for updates. You may even allow the respondents to view the current results when they complete the survey. This works best for one or two-question quick polls rather than longer studies. But knowing they will see the results immediately is frequently a tipping point for respondents who are on the fence about completing your feedback survey form.

This can be helpful in analyzing survey data as well. For example, suppose results for your online survey form vary wildly throughout one week. One day, 10% of respondents use your client’s brand of glass cleaner. The next day, it’s 65%, but then the following day it drops back down to 20%. This could mean something (possibly that you need to check your recipient list) and is good information to track for your client.

The instant results also allow you to add more questions and gain clarification on points you may not have considered prior to launching the survey form. There are dozens of survey web software tools available that can create instant survey results – it’s a method that is worth trying out, but as always, consider your client’s needs before committing to a particular survey method.

Web 2.0 is a term that is frequently tossed around, but there are still many who may not be entirely clear on the definition. The simplest way to boil it down is that Web 1.0 offered little or no opportunities for feedback, while Web 2.0 includes countless ways to “talk back” on the Web. It can include blogs, forums, social media sites, and interactive informational sites like Wikipedia. The users can control the content to an extent on these types of sites, a difference from the static web sites of Web 1.0.

What does this mean for a market researcher? I’ve already discussed methods to use social media for gathering data and survey respondents, but you can use other Web 2.0 sites in a similar manner. It’s a huge topic that I am sure will be revisited again and again.

Web 2.0 sites are good for research purposes. People who would likely make the ideal survey respondents are using these sites to talk back about products. For example, if you were doing a baking survey, the blog Bakerella.com has hundreds of posts and thousands of comments about various recipes and their merits. It is a good start for conducting internet research about the habits of the true baking enthusiast, and could help you in your online questionnaire design.

Despite the opportunities for feedback Web 2.0 offers, it is important to remember that it does not and should not replace more traditional marketing research methods, like electronic surveys, paper questionnaires and other methods. Although respondents are very forthcoming on blogs like Bakerella.com, for a quality study respondents must be properly vetted to ensure they are the correct group.

When designing surveys, researchers often want to find out how people act in addition to how they feel.  Questions about actions and behaviors are especially useful in market research, since you can gain a sense of what customers and consumers are actually doing (as opposed to what they say they’ll do).  However, this area of survey research can be difficult – to the point of being unreliable – so I urge you to follow the guidelines below when asking behavioral questions in your online questionnaire design:

Provide a specific time reference within your survey questions.  This allows for an “anchor” to be set, and can give you a clearer idea of what your respondents do during an average time frame (be it a day or a week).

Make the time frame fairly recent.  It is very difficult for respondents to remember what they did three years ago.  At the very most, stick with questions that reference 12 months ago or less (preferably much less, like the past week or even 24 hours).

Ask only about the respondent’s own behavior.  Unless you’re asking about the actions of small children, it’s best to keep survey questions directed at the individual level rather than for family members or friends.  Asking people about others’ behaviors can provide distorted, unreliable data, as most people overestimate or underestimate certain actions of other people.

Personal and sensitive information should be taken with a grain of salt.  Multiple market research studies have shown the more sensitive a question is, the less likely it is that respondents will answer... and the data provided can be inaccurate, so it is important to consider that people might dance around the absolute truth on these kinds of questions.

It is tempting to include many similar question types with similar response options in your online survey design.  Matrix questions, for example, provide an efficient questionnaire design method to help you gather lots of data in a neat, brief survey form.  It is wise, however, to resist the urge to use too many uniform survey questions and response lists, namely because of two sources of bias that stem from doing so: habituation and acquiescence.

Habituation occurs when respondents begin providing the same answers to survey questions with the same response options.  They start to get in a habit and select the identical response choice for every question.

Acquiescence is related to habituation, and occurs when respondents passively agree with an interviewer or survey questions.  Agree-disagree scales are the most often-used response options in opinion surveys; it is important that you take steps to avoid the chance that respondents will passively agree with your statements in order to quickly complete the questionnaire or provide what they think may be the “right” answers.

To avoid these response biases, you can use online survey software that allows question randomization, break up your matrix questions with other types of questions and scales, and phrase some questions in a manner that makes respondents switch their thinking.  An example of the latter would be to ask a series of positive questions in your survey questionnaire, and then throw in a couple questions worded differently so as not to allow habituation or acquiescence.  Use care up-front in your online questionnaire design to be sure that you'll reduce error and bias in your results.

I think I have always thought of myself as a "planner." While some people "fly by the seat of their pants," I am constantly thinking ahead to the consequences of my actions. The way I see it, this can be a good thing and a bad thing. Being a "planner" is bad because I am not as spontaneous as some of my friends. The "life of the party" is usually not someone who thinks ahead. However, it is a good thing at the same time, especially when operating in the corporate world, and specifically when designing online surveys.

I wrote a post a couple weeks ago about the importance of thinking ahead to reporting before designing a questionnaire online. I want to address a related issue: survey question logic. We on the Cvent Web Surveys Client Services team always strongly encourage our clients to use survey logic to their advantage. Lately, however, I have been getting lots of calls from clients who are having trouble with the logic they have applied. The reason for this is simple: too many clients do not plan or map out the logic they'll apply to their electronic survey. This lack of planning can cause numerous problems, most notably conflicting survey logic.

In the past two weeks, I have spoken with two clients who applied both branch logic and advanced logic to their online surveys. Both of these clients called because they were concerned there was a technical issue which was prohibiting the logic from functioning as it should. "But, I set up my branch logic correctly, and I know that my advanced logic should be working fine too, so what is wrong?" In both cases, the problem was the clients had applied branch logic which conflicted with the advanced logic, and the survey system therefore did not know where to send survey respondents when they answered a question a particular way. "If only these clients had planned ahead and mapped out the survey logic before launching their surveys," I thought, "then this problem could have been avoided."

Moral of the story: Please take advantage of the wonderful survey logic functionality available with Cvent Web Surveys software. However, at the same time, make sure to plan ahead. Even if you are someone who generally "flies by the seat of your pants," when designing online survey questionnaires, you should be a "planner" for once.

In the past several years, many market researchers have shifted to using online surveys more frequently. Traditionally, these online surveys were emailed rather than posted online to avoid data skewing.

When creating the online survey design, there are several things to consider. Make sure the electronic survey is simple and easy to use. Too often, online survey forms are cumbersome or technologically dated. Keep it simple, and you won’t run into any problems.

You must also plan carefully to make sure no questions are missed. You can design online surveys so they will not allow survey respondent to move on if mandatory questions are not completed. In addition to requiring survey questions, I would advise setting up the internet survey so respondents cannot complete the questionnaire more than once. Most online survey research software applications have features to create limits on how many times someone can fill out an online survey form. This prevents skewed data – some respondents will try to complete online survey forms multiple times if there is a giveaway or drawing offered as an incentive.

Make sure you read over your electronic survey a few times as well. Proofreading is especially important when designing online surveys. You must have members of your team proofread not only for grammar and clarity, but also functionality. Make sure all the links work, and the survey can be completed with all your caveats and mandatory questions. 

Online surveys have made it possible to gather information much quicker, and when used properly provide an excellent resource for market researchers.

All too often, we begin the survey creation process, but draw a blank.  How should we design a survey?  Should a survey design have the company look and feel?  Or should we design it to be a fresh look?

Having an online surveys tool that has survey templates built into the system is extremely important.  It eliminates the nagging question, "What is a survey design? How do I create a professional looking web 2.0 survey?"  This gives the survey builder that boost to get the creative juices flowing. In the Cvent Web Survey solution, there are over 50 graphical templates, ready to help you in designing surveys. Any color, any pattern, any style. They are all preloaded in the Cvent Web Surveys application. Definitely check these out the next time you are trying to figure out survey research design ideas for your next questionnaire.

Want to learn more about our survey design software? Sign up for a product demonstration.

Rarely are we asked the question, What is a survey? Typically questions follow the path of, Why do I need a survey program? What am I going to get out of conducting online web surveys to collect customer feedback? Or How do I get started measuring employee satisfaction with online questionnaire templates?

However, sometimes it's good to define market research and what it means, and answer the rarely asked question, What is a survey?

Market Research Definition (mahr-kit-ree-surch)
The gathering and studying of data relating to consumer preferences, purchasing power, etc., especially prior to introducing a product on the market.

Survey Definition (ser-vey)
Collect quantitative information about items in a population. Surveys of human populations and institutions are common in political polling and government, health, social science and marketing research.

Chances are though, you're still wondering about those other questions. These posts may help you answer those ever burning questions about why you should create and design surveys.

Yes, you read right; we’re not talking about cloning people. But wouldn’t it be awesome if you could clone your electronic surveys? The advantages are endless. If you have an internet survey that has to be sent out to people who attended different conferences on different dates, the headers and welcome pages would be different – as would the responses coming in, but the questions and emails would be almost exactly the same. Or if there’s an old online survey, closed and archived years ago, but you’ve realized you need to send it out again – clone it!

The best part about cloning surveys is that you can still make changes to them – so if you were designing surveys with similar content for five different groups of people, you could create a basic survey with all the questions, headers, templates, security settings, etc. and make minor changes to each of them so that each group gets a customized online web survey, but you only spend a fifth of the time.

When you clone a survey form, all information except the people added to its targeted list, gets carried over. So the next time you spend hours adding complicated branch and pipe survey question logic to an online questionnaire, worrying all the time that you will have to do it again and again, you can breathe a little easy. You always have the option to clone a survey irrespective of whether it’s in Test mode, Active, Closed or even Archived. Once you’ve done the work, you will never have to do it again.

So go ahead and clone your next survey – send it out to different people, generate different reports for each of them – you can even have one as anonymous and the other collecting contact information from respondents. As for cloning people, that debate will rage on.

Just because you find correlation in your data when analyzing survey results does not mean there is causation. I find this is a common mistake in lots of survey reports when someone is new to survey research or conducting data analysis. Take this example:


Some people see this connection as a causation. Customers are satisfied because you contact them frequently to make sure everything is going well. The problem is, it's not a causation. Causation are extremely hard to prove because you cannot control every factor. For example, you may split your territory by industry and your solution suits some industries better than others. Or clients who are really satisfied simply use the product more often so the customer success managers reach out to them more frequently, because they are more likely to have questions. While those who use the product less have less to be satisfied about and may feel they are wasting those budget dollars.

To be able to prove causation, you need to be able to rule out all other possible explanations for the connection. As you can imagine, that's almost impossible to do since we do not control outside factors influencing the survey respondent or even the greater survey sample. Instead, when situations like these occur, we're seeing a correlation between two things. In my customer survey question examples, there's a correlation between how satisfied customers and how often they are contacted.

This principle does not only apply to customer survey research, it also applies to analyzing employee feedback forms, product surveys, market research and any other type of data collection and analysis.