Sample Online Survey

As is the case with any research method, there are advantages and disadvantages to using an online web survey to collect data about customers, employees or the public-at-large.  For example, online surveys offer you the ability to gather vast amounts of data from many respondents at the same time, get your data back in an electronic form, see real-time results and automate analysis/reporting tasks... and you can do all of this affordably.  You can even use a relatively small research survey sample to accurately estimate the opinions of your larger population (for survey research in general).

But in some cases, it is best to use a “mixed-methods” approach to your research project.  This means you combine the online survey method with another kind of investigation, such as interviews or focus groups, in order to produce more well-rounded data and conclusions.  Here are some examples of when a mixed-methods approach is likely better than an online survey form by itself:

A survey conducted by research firm BIA/Kelsey found that 9% of small and midsize business use Twitter for marketing purposes. An earlier survey by online marketing resource, MarketingProfs, showed that 84% of small businesses expect they will increase their use of Twitter and other social media outlets in the future. That same report also takes a look at how large businesses - including Dell and Disney - are using Twitter to reach devoted fans and potential customers. Companies large and small have begun using Twitter. I find this list of companies, although old, to be quite impressive!

At Cvent Web Surveys, we often recommend the use of Twitter, Facebook, and other social media outlets to reach a wider audience for your web survey. We've often found it to be most useful in monitoring and acted fast to ensure customer satisfaction. Bad news travels fast these days, and you have to be ready to react. 

If you are distributing your electronic survey in multiple channels (social media, your public-facing website, and through Cvent's email survey software platform) it is important to track where people are coming from. By analyzing where your respondents are coming from, you can more efficiently distribute your surveys in the future or weight responses. This is especially important if you are paying to advertise your survey - perhaps on traditional banner display advertising, website pop-ups, or search ads such as Google AdWords.

Cvent Web Surveys software provides two ways of tracking and analyzing this data. First, we have our "Reference ID". Reference IDs are a short string of text added to the end of your survey URL – "twitter", "yelp", or "google" – for instance, to allow you to track what other website "referred" the survey respondent to complete your online survey. These strings of text will then allow you to use the Cvent's built-in survey reporting to see, analyze, and slice and dice the information to give a clear view of where the survey respondents are coming from. 

Second, Cvent was proud to announce in our last quarterly product enhancement, the addition of Google Analytics integration. Google Analytics is Google's free tool for tracking visitors to your websites. The integration with Cvent Web Surveys software is extremely simple to set up and is a fantastic addition for our clients who use and love Google Analytics.

Now while I heartily recommend using social media as a tool to get your survey in front of more people, there is one thing that is crucial to remember: many businesses block social media websites. Robert Half International, an IT recruiting firm, reported 54% of companies completely block access to sites such as Twitter, Facebook, and even business-oriented LinkedIn. This means, if you're trying to attract those of us who work, you may be facing a higher hurdle than you imagined.

One of the best descriptors attributed to good survey questions is the word specific.  A main goal in designing valid, reliable survey questions is doing everything in your power to make them clear, standardized and unambiguous.  A great way to follow through on that is to make sure your questions are as specific as possible.  The degree of specificity affects how people interpret and respond to your survey questions.  Several examples are listed below to help you analyze your survey questions regarding their level of specificity.

Be clear with demographic questions.  Your objective here should be to use words and categories that your respondents can clearly understand and identify with.  Will you use Census designations for Ethnicity?   Exactly what do you mean by “marital status?”

Define vague concepts, words and phrases.  Your respondents will likely widely differ with regard to their backgrounds, experiences and perceptions.  Words like “justice” and “equality” can mean very different things to people, so it is best to define exactly what you are asking about.

Objective or subjective?  Use verbs that trigger respondents.  Attitude survey questions are usually subjective (i.e., how do you feel about the war?), while behavioral questions are usually objective (i.e., what did you eat for supper last night?).

Always try to attach a time frame to behavioral and recall questions.

For market and product research, identify the actual brand name, and ask how they feel about specific items, not just groups or genres.  For example, don’t use the vague word “furniture” if you’re really interested in how they feel about a table or a chair.

Survey designing software can't look at your questions and tell you if your online survey form will give you the data you want. However, it does make it easier when you're designing a web survey because it takes out the added complications with writing your own code. So make sure next time you're working on an instant survey form, you think about the specificity of your survey questions.

When you plan your online market research study, you still need to remember the standard steps for any market research project, plus some online specific tasks:


The time frame for an online study is, as discussed previously, often shorter because of the immediacy of the results. Keep this in mind but be sure to give yourself enough time to analyze survey responses.

Every year it seems like the holidays get earlier and earlier. This year I noticed stores had holiday decorations out before Halloween costumes! While I can attest to retail stores being ready for the holiday shopping season, it makes me wonder, are you?

Online web surveys are just one more way you have the ability to engage with your consumer email lists. Customer surveys should be no brainers. But how can an online survey form apply to holiday marketing strategies? Well that depends, are you setting survey objectives that will impact your marketing strategies?

Here's what I mean: I receive countless retail marketing emails everyday. Most of which I opted in to, however, no one ever asks me about my shopping habits or preferences. These companies could easily put together a retail survey using an online survey application and find out more about who I am as a customer, what I prefer, how frequently I shop in their stores or how frequently I visit their website. From the responses they get back, they can then customize their marketing messages based on how I shop with them. If I only shop online, it may make sense to group these customers together and do a special online promotion. All the customers who only shop in the stores would receive a different promotional offer because they shop differently than I do.

This concept goes back to one I've shared previously: profiling customers to better understand your organization. The holidays are closing in quickly, but it's not too late to build an online survey and collect customer feedback instantly.

How have you designed online survey questionnaires in the past to impact your marketing strategies?

Too often companies make the common mistake of surveying customers after it’s too late. Many will use customer satisfaction surveys to gauge how they did last month or last quarter. In truly proactive, customer-focused organizations customer feedback is used on a daily basis to increase revenue and improve customer satisfaction and loyalty when the timing is critical for the customer.

Most customer feedback collected is of immediate importance to them and requires immediate attention. Waiting until a survey closes to attend to customer needs is a recipe for disaster. This could include missing out on hot leads for new business, up-sell and cross-sell opportunities, or worse not reaching out to identify the dissatisfaction from an irate customer.


Your customer is seeing how much you really care about their business. Why should they waste their time taking a web survey form to help you improve, if there is clearly no actions being taken to apply the feedback you’ve received? Why is a customer going to work with a firm that they feel is wasting their time? Now the customer is not only feeling as if you aren’t concerned about their needs and opinions, you’ve pushed them to look at your competitors. The goal is to take this opportunity to show the customer how much you truly care about their needs.

Having a system in place to streamline and automate your produce with real time alerts will allow you to serve you customers in a more timely fashion, keeping them happy and better yet loyal.

One thing that I think people often overlook when picking out online survey software is the team behind the product. Here at Cvent Web Surveys, we take pride in our customer care team. With today’s blog post, I wanted to share some great news – Cvent’s Client Services team is the winner of the 2009 International Service Excellence Award for Contact Center by the Customer Service Institute of America (CSIA)!

The CSIA is the body delegated by the International Council of Customer Service Organizations (ICCSO) to manage the International Service Excellence Awards. These awards are the world’s peak customer service awards with leading customer service organizations and individuals around the globe being nominated in a variety in categories to recognize their commitment customer service excellence.

We are proud and honored to have received this award. Having been a member of the Cvent Web Surveys Client Services team, I can attest to the level of service we provide.

Cvent is not just a survey software company – we are consultative, and pledge every effort to your survey success and satisfaction. We want our clients to get the data they need, in the way they need to see it. We work with our clients – hand in hand – to ensure no question goes unturned!

To read more about Cvent’s Support and Training, visit our website. Cvent’s Client Services team is available by phone and email 24 x 5 – and we offer weekend support hours as well! In fact, 90% of calls into our client services line are answered in less than 90 seconds, and we have the highest ratio of service personnel to clients in industry.

When choosing an online survey application, customer care cannot be overlooked. In this economy, utilize expert advice to increase responses and data quality and demand more from your survey partner!

Surveys are an invaluable tool for researching the community attitudes, employee concerns, product needs, customer loyalty and priorities held by different groups or target audiences.  Designing a questionnaire and collecting survey responses from a sample allows us to draw a profile of the group as a whole, and perhaps perform some correlation analysis to understand the source of those feelings.  The online survey findings can then support fact-based organizational decisions or improvement projects to help continually improve the organization over time.

Survey research can be applied to many venues.  Here are just a few practical applications listed below:

An Internal Employee Survey could identify reasons for low employee retention and provide ideas for reducing those costs, such as a better designed benefit program, improved training opportunities, or problems in the way the organization functions.

A Training Survey can identify how a training program has improved the capabilities of some group and how the training program itself can be improved.  

A Product Satisfaction Survey can identify initial customer experiences with a product, providing data to address unforeseen problems and help the next product release.  

A Market Research Survey can identify customers needs when creating these new service and product offerings.  Surveys can be part of Design for Six Sigma activities.  

An Association Survey, which is similar to market research and customer surveys, can show the member benefits most of interest.

However, a survey program is only valuable if it is properly designed and executed.  While performing a survey project seems deceptively simple – it's just a bunch of questions, and survey software tools make electronic surveys quick and cheap – a small mistake in the survey questionnaire design or survey administration can skew or bias the data, leading to erroneous conclusions.  No organization should ever make critical business decisions based on unreliable or invalid data.

Bad data is worse than no data!

There is sufficient evidence from prior studies suggesting it is a good idea to keep respondents informed of their progress during internet surveys.  Respondents may suffer from fatigue or irritation, and may abandon web survey forms – even if they have only a few questions left to complete the questionnaire.  This may lead researchers and project managers to adjust their online questionnaire design by reordering questions to include “important” or sensitive items earlier, possibly causing more survey respondents to abandon mid-stream.

There are various survey design techniques that can be employed to keep respondents informed of their progress, especially within electronic surveys.  One way is to design web surveys so the entire survey web form can be viewed on a single page; but while this allows respondents to scroll down and see the total length of a questionnaire, this setup is less than ideal. 

Another method is to include words or symbols in headers and transitions that notify progress (such as section numbers, the words "next" and "finally," etc.)  By far however, the most popular and effective method of keeping respondents in the loop is to include a progress bar. 

A progress bar is a graphic or set of words that let respondents know how far along they are in the survey process.  Progress bars are especially useful for shorter, instant surveys, since answering only a few questions will show that they are through a significant portion of the questionnaire.

Cvent Web Surveys software makes it easy to include a progress bar as part of an effective online survey design.  You can even select among three options (percent complete, page x of y, or a bar graphic that fills as respondents move through the survey).

I realized it only after the emails had been sent.  It was a test survey that wasn’t intended for entry level employees and senior management.  It was an employee evaluation that had initially been slated to go out only to our newest batch of hires.  Only last minute did we discover we could use logic to create one interactive survey online for senior management to evaluate the progress of new hire training, and new hire survey questions for them to evaluate the training process.  The wording, the slang, everything was wrong!  What did I do…

Hopefully, this scenario has appeared solely in a nightmare (and yes, survey nightmares are very real and very terrifying).  But a similar, less daunting situation is common among many: You have an online web survey that will go out to two groups of respondents, the wording in the two would ideally be completely different, and you do not have the time to go into the two templates you’ve created and select the different audiences then manually send the emails.  Not to mention, this method would require using Cvent’s email survey tool's default templates and the Custom Messages.  What if I told you it was possible to create entirely separate email marketing campaigns, set the emails up to go out automatically AND keep the both default templates and custom messages for each group of respondents? 

The multiple email marketing campaigns feature allows you to do just this!  You can select the number of separate email blasts you intend on sending, you can add entirely different contact groups to each campaign, and you can set up your emails to go out automatically. 

1. This feature is perfect if you have a multilingual survey.  You can translate your message into as many languages as you like to ensure maximum comprehension. 

2. Multiple email campaigns is also great for newsletters that are tied to a generic instant survey.  Instead of copying your survey and creating a new email blast every month, you can add an email campaign for the October '09 Newsletter, November '09 Newsletter, December '09 Newsletter and so forth.  You can then get an aggregate view of the data collected across all months.

3. The nightmare scenario outlined above.  It is likely that emails sent to your senior management are not worded the same as emails sent to entry-level new hires.  The formality of the verbiage aside, your new hires might be a bit disconcerted to know that the bosses of the bosses are watching them and evaluating their decisions. 

We all know creating surveys and designing questionnaires can be a bit overwhelming.  Why put yourself through that again just so different people can receive the same questionnaire?

I’ve talked extensively in the past about using Facebook for market research. But there are other Web 2.0 sites that you can use effectively as well.

For example, Twitter is growing more and more. You can search on Twitter for keywords or competing products to see how often they are mentioned, and what people are saying. This is a quick and simple method of gathering background info.

There are many sites where people mark things they like or dislike, like digg and del.icio.us. Everything on digg is submitted by users and then ranked by users. After a story has enough “diggs” it is listed on the front page. You can search for your client and see what types of stories are making their way to the site and get an idea of what kind of buzz is surrounding them. Del.icio.us allows users to tag and bookmark their favorite pages, and then allows all users to access rankings. You can find out how many people have bookmarked your client and competitors.

Of course, both these groups are by their nature narrower than most studies require. Results for each page are primarily involving only regular users of the site. You should broaden your search to find other respondents so your data is more accurate.

Web 2.0 sites are great places to find information and do background research. They also tend to be good places to find electronic survey respondents, as users are web savvy and opinionated. But as I mentioned, be sure not to make your respondent range too narrow.

In yesterday's post we looked at how your survey question items, if worded improperly, may be biased, resulting in unusable or incorrect results. Now we know how survey questions shouldn't look, let's delve deeper into why those previous examples were poorly worded, and how with just a few small adjustments you can easily start to hone question items within your survey forms.

When you design internet surveys, how do you write objective questions that won't bias your results one way or the other?

Here are a few main points:


These first few points are a great starting place when you're designing a questionnaire. The final post in this series will look at a few final survey best practice tips for writing great survey questions and the benefits you'll gain from crafting quality items. In the meantime, questions previously stored in our question library already come properly worded and ready for use!

Marketing departments love client testimonials and customer quotes. They're great to share with the sales team when prospects need referrals, they're helpful to put in powerpoint presentations, and can be a critical piece of any marketing website. Getting those sound bites from clients can be a challenge, however. As you can imagine, the challenge only gets bigger if your organization sells consumer products. Admittedly, the growth of social media has made finding people who are saying good things about your product, services or organization has gotten a little bit easier. But, it could be easier still.

Imagine a world where you had a plethora of client sound bites and quotes to share with the outside world. A marketers dream. No more tracking down sales reps to talk to their clients and mangle their words. Everything is already done for you. Well, if you're conducting client satisfaction surveys or consumer surveys, you should be gathering those sound bites already. You can even consider using website polls and website usability surveys to get those sound bites. Depending on the online questionnaire design, you may still need to contact the customer to get permission to use their quote, but the hard part is still done. You have the quote.

Question remains, how do you pull these customer quotes out of your survey web form? You make sure to add an additional comment box. Customers who love you will typically share those sentiments in the additional comments area, particularly if there are not other open ended questions in the survey for them to share why they like an organization.

Still wondering why your marketing department needs sound bites? Opinions for others is one of the driving forces behind purchasing decisions. If you want to learn more about the importance of customer reviews, check out this past post. It's also worth mentioning, this same idea can work for employee surveys and your HR department for recruiting purposes.

When it comes to marketing, understanding what information your audience wants to see is very important. In the Cvent survey management system, there is an option for Click Tracking. This part of the email marketing tool is great.

Here is how it works: When you create an eNewletter in Cvent, you can embed active links. These links can then be tracked through the Cvent Web Surveys reporting engine.  Unbelievable. Imagine being able to see what articles are important to your readers by running a simple report.  This gives you the insight needed to target specific groups more effectively.

It is important not to post a survey online publicly if you need a response from a specific target group. There are several ways to do this. You can, and should, make the survey web site link a "hidden URL," meaning it's not linked anywhere else on your web site and not found through Internet searches.

Include the link only in the email to the targeted group. You can ensure survey respondents only respond once by using the survey security settings within your online survey software application. If you are concerned about the content of your survey being copied, it is simple enough to lock the code to prevent copying and pasting. It goes without saying that someone who is determined to copy the content will find a means to do so, but there’s no point in making it easy for them!

If you are afraid that content of your online web survey will leak, you need to take every precaution to prevent it. Selecting a web survey company who houses their servers in a SAS-70 environment will give you the highest level of security. If security is important to your organization or your client, make sure to ask your survey company about data security. There may be times when a client wants a completely secret survey, and therefore online surveying may not be the best choice. However, there are high-level security features like encrypted connections if you decide to forge ahead with an online survey. For most surveys online, the standard security precautions will prove more than enough for your purposes.

Designing a survey instrument can be one of the most fun aspects of conducting survey research. In this two-part tutorial, we'll first look at some of the "Don'ts" of writing survey questions, and then we will look at some of the "Do's".

Once the brainstorming of ideas phase has been accomplished and you have a good idea what the scope of your research is, it's time to get down to putting question ideas on paper (or online!).

However creating good survey questions can turn into quite an art form, as we all struggle to word our items in such a way that our participants will understand what information we are asking for. Recognizing when you've accidentally designed a survey question that is worded in a non-objective way is the sign of a good researcher. Let's see if you can spot the poorly worded question from this group of sample survey questions:


Now that we've looked at some question items that shouldn't be put into surveys, next time we can define some ways we can create objective question items that will provide us with the results we need!

In a recent post, I described the differences between nonprobability and probability sampling methods in online survey designs.  Probability methods are preferred if at all possible, because they allow you to make generalizations from your electronic survey results to a larger population or target audience.  One kind of probability-based sampling technique is called systematic random sampling. 

To employ a systematic random sampling design for your online web survey, you first select a case at random from your exhaustive population list, and then select further cases at identical intervals, determined by how many people you want to sample in total.  If you wanted to sample ten people from a population list of 150, you would then choose every fifteenth person after selecting someone in the first 15 cases (to ensure you will select 10 people in total).

This provides an easy way to obtain a random sample of your population list or sampling frame, because as long as your data is ordered randomly, you can begin simply by selecting any record or case and go from there.  This is an important caveat though: your records must be randomly ordered for a systematic sample to be effective. 

Take this example of survey sampling, let’s say you have a sampling frame (list) of people that is currently ordered alphabetically by last name, and you are interested in subgroup analyses by ethnicity.  It would be wise in this case to rearrange the records into a truly random order (i.e., not alphabetically), because last names from certain backgrounds may be more likely to begin with a particular letter.  While systematic sampling provides an easy way to generate a random sample for online surveys, you do need to be sure there is no hidden order within your population list or sampling frame.

Some people still believe in using paper surveys instead of designing online questionnaires, sending personalized email survey reminders and utilizing electronic survey methods. Sometimes I have to wonder, Why? Some of the reasons people claim they want to use paper questionnaires instead of web survey forms is because they think it's cheaper. That's not usually the case.

Consider this: Online survey response rates are typically hire than response rates on a direct mail survey. Online survey research designed well will yeild response rates from 20-40%. Let's be conservative and say the paper survey you plan to send out will get a 20% response rate as well. I looked up the cost of a ream of standard multipurpose paper (500 sheets) on Office Depot's website: $7.09. A black ink cartridge for a laser printer is $77.99 and will print on a maximum of 2,200 pages. Most printed surveys will be kept to three to five pages, longer than that your survey form looks over whelming from the very beginning and is going to lower your response rate. This is how that cost quickly adds up:


Keep in mind these costs don't include the staff time of stuffing envelopes to send out the direct mail survey, or the staff time of manually entering in all of the responses into some sort of data collection software, be it Excel or a survey software tool. You can figure it will take an intern a week to send out 5,000 survey responses and another week for them to do the data entry for the 1,000 responses. So that's an additional cost of $600 to $800 for one survey campaign. What if you want to do a quarterly customer satisfaction survey? You're looking at an annual cost of over $25,000.

For me, that would be enough reason alone to look into an online survey software application. With a web survey tool, you have the ability to not only send one invitation, but reminders survey emails as well. Sending reminders, as we've mentioned before, can drastically increase your survey response rate. My suggestion: don't fall for the myth that paper surveys are cheaper than online surveys. It's definitely the other way around.

There advantages and disadvantages to conducting web or electronic surveys as opposed to traditional survey modes such as personal interviewing, telephone and mail.  One of the advantages of an online survey design is a possible reduction of what is known as social desirability bias. 

Social desirability bias occurs when survey respondents offer responses that portray them in a positive or more favorable manner to others. 

When a face-to-face interviewer asks personal or sensitive questions respondents feel have a “good answer” and a “bad answer” (such as criminal behavior), they may underreport bad behavior and overreport good behavior, for example.  While this bias may be reduced in online surveys due to the absence of an interviewer, there are topics that may produce invalid or unreliable data, regardless of survey mode.  This is something to consider before finalizing your online questionnaire design or web survey forms.

Survey questions within the following content areas are especially subject to social desirability in a survey form (i.e., respondents believe particular responses are “better” than others). 

Just because you find correlation in your data when analyzing survey results does not mean there is causation. I find this is a common mistake in lots of survey reports when someone is new to survey research or conducting data analysis. Take this example:


Some people see this connection as a causation. Customers are satisfied because you contact them frequently to make sure everything is going well. The problem is, it's not a causation. Causation are extremely hard to prove because you cannot control every factor. For example, you may split your territory by industry and your solution suits some industries better than others. Or clients who are really satisfied simply use the product more often so the customer success managers reach out to them more frequently, because they are more likely to have questions. While those who use the product less have less to be satisfied about and may feel they are wasting those budget dollars.

To be able to prove causation, you need to be able to rule out all other possible explanations for the connection. As you can imagine, that's almost impossible to do since we do not control outside factors influencing the survey respondent or even the greater survey sample. Instead, when situations like these occur, we're seeing a correlation between two things. In my customer survey question examples, there's a correlation between how satisfied customers and how often they are contacted.

This principle does not only apply to customer survey research, it also applies to analyzing employee feedback forms, product surveys, market research and any other type of data collection and analysis.