Online Questionnaires

One of the best descriptors attributed to good survey questions is the word specific.  A main goal in designing valid, reliable survey questions is doing everything in your power to make them clear, standardized and unambiguous.  A great way to follow through on that is to make sure your questions are as specific as possible.  The degree of specificity affects how people interpret and respond to your survey questions.  Several examples are listed below to help you analyze your survey questions regarding their level of specificity.

Be clear with demographic questions.  Your objective here should be to use words and categories that your respondents can clearly understand and identify with.  Will you use Census designations for Ethnicity?   Exactly what do you mean by “marital status?”

Define vague concepts, words and phrases.  Your respondents will likely widely differ with regard to their backgrounds, experiences and perceptions.  Words like “justice” and “equality” can mean very different things to people, so it is best to define exactly what you are asking about.

Objective or subjective?  Use verbs that trigger respondents.  Attitude survey questions are usually subjective (i.e., how do you feel about the war?), while behavioral questions are usually objective (i.e., what did you eat for supper last night?).

Always try to attach a time frame to behavioral and recall questions.

For market and product research, identify the actual brand name, and ask how they feel about specific items, not just groups or genres.  For example, don’t use the vague word “furniture” if you’re really interested in how they feel about a table or a chair.

Survey designing software can't look at your questions and tell you if your online survey form will give you the data you want. However, it does make it easier when you're designing a web survey because it takes out the added complications with writing your own code. So make sure next time you're working on an instant survey form, you think about the specificity of your survey questions.

Every year it seems like the holidays get earlier and earlier. This year I noticed stores had holiday decorations out before Halloween costumes! While I can attest to retail stores being ready for the holiday shopping season, it makes me wonder, are you?

Online web surveys are just one more way you have the ability to engage with your consumer email lists. Customer surveys should be no brainers. But how can an online survey form apply to holiday marketing strategies? Well that depends, are you setting survey objectives that will impact your marketing strategies?

Here's what I mean: I receive countless retail marketing emails everyday. Most of which I opted in to, however, no one ever asks me about my shopping habits or preferences. These companies could easily put together a retail survey using an online survey application and find out more about who I am as a customer, what I prefer, how frequently I shop in their stores or how frequently I visit their website. From the responses they get back, they can then customize their marketing messages based on how I shop with them. If I only shop online, it may make sense to group these customers together and do a special online promotion. All the customers who only shop in the stores would receive a different promotional offer because they shop differently than I do.

This concept goes back to one I've shared previously: profiling customers to better understand your organization. The holidays are closing in quickly, but it's not too late to build an online survey and collect customer feedback instantly.

How have you designed online survey questionnaires in the past to impact your marketing strategies?

Too often companies make the common mistake of surveying customers after it’s too late. Many will use customer satisfaction surveys to gauge how they did last month or last quarter. In truly proactive, customer-focused organizations customer feedback is used on a daily basis to increase revenue and improve customer satisfaction and loyalty when the timing is critical for the customer.

Most customer feedback collected is of immediate importance to them and requires immediate attention. Waiting until a survey closes to attend to customer needs is a recipe for disaster. This could include missing out on hot leads for new business, up-sell and cross-sell opportunities, or worse not reaching out to identify the dissatisfaction from an irate customer.


Your customer is seeing how much you really care about their business. Why should they waste their time taking a web survey form to help you improve, if there is clearly no actions being taken to apply the feedback you’ve received? Why is a customer going to work with a firm that they feel is wasting their time? Now the customer is not only feeling as if you aren’t concerned about their needs and opinions, you’ve pushed them to look at your competitors. The goal is to take this opportunity to show the customer how much you truly care about their needs.

Having a system in place to streamline and automate your produce with real time alerts will allow you to serve you customers in a more timely fashion, keeping them happy and better yet loyal.

One thing that I think people often overlook when picking out online survey software is the team behind the product. Here at Cvent Web Surveys, we take pride in our customer care team. With today’s blog post, I wanted to share some great news – Cvent’s Client Services team is the winner of the 2009 International Service Excellence Award for Contact Center by the Customer Service Institute of America (CSIA)!

The CSIA is the body delegated by the International Council of Customer Service Organizations (ICCSO) to manage the International Service Excellence Awards. These awards are the world’s peak customer service awards with leading customer service organizations and individuals around the globe being nominated in a variety in categories to recognize their commitment customer service excellence.

We are proud and honored to have received this award. Having been a member of the Cvent Web Surveys Client Services team, I can attest to the level of service we provide.

Cvent is not just a survey software company – we are consultative, and pledge every effort to your survey success and satisfaction. We want our clients to get the data they need, in the way they need to see it. We work with our clients – hand in hand – to ensure no question goes unturned!

To read more about Cvent’s Support and Training, visit our website. Cvent’s Client Services team is available by phone and email 24 x 5 – and we offer weekend support hours as well! In fact, 90% of calls into our client services line are answered in less than 90 seconds, and we have the highest ratio of service personnel to clients in industry.

When choosing an online survey application, customer care cannot be overlooked. In this economy, utilize expert advice to increase responses and data quality and demand more from your survey partner!

Surveys are an invaluable tool for researching the community attitudes, employee concerns, product needs, customer loyalty and priorities held by different groups or target audiences.  Designing a questionnaire and collecting survey responses from a sample allows us to draw a profile of the group as a whole, and perhaps perform some correlation analysis to understand the source of those feelings.  The online survey findings can then support fact-based organizational decisions or improvement projects to help continually improve the organization over time.

Survey research can be applied to many venues.  Here are just a few practical applications listed below:

An Internal Employee Survey could identify reasons for low employee retention and provide ideas for reducing those costs, such as a better designed benefit program, improved training opportunities, or problems in the way the organization functions.

A Training Survey can identify how a training program has improved the capabilities of some group and how the training program itself can be improved.  

A Product Satisfaction Survey can identify initial customer experiences with a product, providing data to address unforeseen problems and help the next product release.  

A Market Research Survey can identify customers needs when creating these new service and product offerings.  Surveys can be part of Design for Six Sigma activities.  

An Association Survey, which is similar to market research and customer surveys, can show the member benefits most of interest.

However, a survey program is only valuable if it is properly designed and executed.  While performing a survey project seems deceptively simple – it's just a bunch of questions, and survey software tools make electronic surveys quick and cheap – a small mistake in the survey questionnaire design or survey administration can skew or bias the data, leading to erroneous conclusions.  No organization should ever make critical business decisions based on unreliable or invalid data.

Bad data is worse than no data!

There is sufficient evidence from prior studies suggesting it is a good idea to keep respondents informed of their progress during internet surveys.  Respondents may suffer from fatigue or irritation, and may abandon web survey forms – even if they have only a few questions left to complete the questionnaire.  This may lead researchers and project managers to adjust their online questionnaire design by reordering questions to include “important” or sensitive items earlier, possibly causing more survey respondents to abandon mid-stream.

There are various survey design techniques that can be employed to keep respondents informed of their progress, especially within electronic surveys.  One way is to design web surveys so the entire survey web form can be viewed on a single page; but while this allows respondents to scroll down and see the total length of a questionnaire, this setup is less than ideal. 

Another method is to include words or symbols in headers and transitions that notify progress (such as section numbers, the words "next" and "finally," etc.)  By far however, the most popular and effective method of keeping respondents in the loop is to include a progress bar. 

A progress bar is a graphic or set of words that let respondents know how far along they are in the survey process.  Progress bars are especially useful for shorter, instant surveys, since answering only a few questions will show that they are through a significant portion of the questionnaire.

Cvent Web Surveys software makes it easy to include a progress bar as part of an effective online survey design.  You can even select among three options (percent complete, page x of y, or a bar graphic that fills as respondents move through the survey).

I realized it only after the emails had been sent.  It was a test survey that wasn’t intended for entry level employees and senior management.  It was an employee evaluation that had initially been slated to go out only to our newest batch of hires.  Only last minute did we discover we could use logic to create one interactive survey online for senior management to evaluate the progress of new hire training, and new hire survey questions for them to evaluate the training process.  The wording, the slang, everything was wrong!  What did I do…

Hopefully, this scenario has appeared solely in a nightmare (and yes, survey nightmares are very real and very terrifying).  But a similar, less daunting situation is common among many: You have an online web survey that will go out to two groups of respondents, the wording in the two would ideally be completely different, and you do not have the time to go into the two templates you’ve created and select the different audiences then manually send the emails.  Not to mention, this method would require using Cvent’s email survey tool's default templates and the Custom Messages.  What if I told you it was possible to create entirely separate email marketing campaigns, set the emails up to go out automatically AND keep the both default templates and custom messages for each group of respondents? 

The multiple email marketing campaigns feature allows you to do just this!  You can select the number of separate email blasts you intend on sending, you can add entirely different contact groups to each campaign, and you can set up your emails to go out automatically. 

1. This feature is perfect if you have a multilingual survey.  You can translate your message into as many languages as you like to ensure maximum comprehension. 

2. Multiple email campaigns is also great for newsletters that are tied to a generic instant survey.  Instead of copying your survey and creating a new email blast every month, you can add an email campaign for the October '09 Newsletter, November '09 Newsletter, December '09 Newsletter and so forth.  You can then get an aggregate view of the data collected across all months.

3. The nightmare scenario outlined above.  It is likely that emails sent to your senior management are not worded the same as emails sent to entry-level new hires.  The formality of the verbiage aside, your new hires might be a bit disconcerted to know that the bosses of the bosses are watching them and evaluating their decisions. 

We all know creating surveys and designing questionnaires can be a bit overwhelming.  Why put yourself through that again just so different people can receive the same questionnaire?

I’ve talked extensively in the past about using Facebook for market research. But there are other Web 2.0 sites that you can use effectively as well.

For example, Twitter is growing more and more. You can search on Twitter for keywords or competing products to see how often they are mentioned, and what people are saying. This is a quick and simple method of gathering background info.

There are many sites where people mark things they like or dislike, like digg and del.icio.us. Everything on digg is submitted by users and then ranked by users. After a story has enough “diggs” it is listed on the front page. You can search for your client and see what types of stories are making their way to the site and get an idea of what kind of buzz is surrounding them. Del.icio.us allows users to tag and bookmark their favorite pages, and then allows all users to access rankings. You can find out how many people have bookmarked your client and competitors.

Of course, both these groups are by their nature narrower than most studies require. Results for each page are primarily involving only regular users of the site. You should broaden your search to find other respondents so your data is more accurate.

Web 2.0 sites are great places to find information and do background research. They also tend to be good places to find electronic survey respondents, as users are web savvy and opinionated. But as I mentioned, be sure not to make your respondent range too narrow.

In yesterday's post we looked at how your survey question items, if worded improperly, may be biased, resulting in unusable or incorrect results. Now we know how survey questions shouldn't look, let's delve deeper into why those previous examples were poorly worded, and how with just a few small adjustments you can easily start to hone question items within your survey forms.

When you design internet surveys, how do you write objective questions that won't bias your results one way or the other?

Here are a few main points:


These first few points are a great starting place when you're designing a questionnaire. The final post in this series will look at a few final survey best practice tips for writing great survey questions and the benefits you'll gain from crafting quality items. In the meantime, questions previously stored in our question library already come properly worded and ready for use!

Marketing departments love client testimonials and customer quotes. They're great to share with the sales team when prospects need referrals, they're helpful to put in powerpoint presentations, and can be a critical piece of any marketing website. Getting those sound bites from clients can be a challenge, however. As you can imagine, the challenge only gets bigger if your organization sells consumer products. Admittedly, the growth of social media has made finding people who are saying good things about your product, services or organization has gotten a little bit easier. But, it could be easier still.

Imagine a world where you had a plethora of client sound bites and quotes to share with the outside world. A marketers dream. No more tracking down sales reps to talk to their clients and mangle their words. Everything is already done for you. Well, if you're conducting client satisfaction surveys or consumer surveys, you should be gathering those sound bites already. You can even consider using website polls and website usability surveys to get those sound bites. Depending on the online questionnaire design, you may still need to contact the customer to get permission to use their quote, but the hard part is still done. You have the quote.

Question remains, how do you pull these customer quotes out of your survey web form? You make sure to add an additional comment box. Customers who love you will typically share those sentiments in the additional comments area, particularly if there are not other open ended questions in the survey for them to share why they like an organization.

Still wondering why your marketing department needs sound bites? Opinions for others is one of the driving forces behind purchasing decisions. If you want to learn more about the importance of customer reviews, check out this past post. It's also worth mentioning, this same idea can work for employee surveys and your HR department for recruiting purposes.

When it comes to marketing, understanding what information your audience wants to see is very important. In the Cvent survey management system, there is an option for Click Tracking. This part of the email marketing tool is great.

Here is how it works: When you create an eNewletter in Cvent, you can embed active links. These links can then be tracked through the Cvent Web Surveys reporting engine.  Unbelievable. Imagine being able to see what articles are important to your readers by running a simple report.  This gives you the insight needed to target specific groups more effectively.

It is important not to post a survey online publicly if you need a response from a specific target group. There are several ways to do this. You can, and should, make the survey web site link a "hidden URL," meaning it's not linked anywhere else on your web site and not found through Internet searches.

Include the link only in the email to the targeted group. You can ensure survey respondents only respond once by using the survey security settings within your online survey software application. If you are concerned about the content of your survey being copied, it is simple enough to lock the code to prevent copying and pasting. It goes without saying that someone who is determined to copy the content will find a means to do so, but there’s no point in making it easy for them!

If you are afraid that content of your online web survey will leak, you need to take every precaution to prevent it. Selecting a web survey company who houses their servers in a SAS-70 environment will give you the highest level of security. If security is important to your organization or your client, make sure to ask your survey company about data security. There may be times when a client wants a completely secret survey, and therefore online surveying may not be the best choice. However, there are high-level security features like encrypted connections if you decide to forge ahead with an online survey. For most surveys online, the standard security precautions will prove more than enough for your purposes.

Designing a survey instrument can be one of the most fun aspects of conducting survey research. In this two-part tutorial, we'll first look at some of the "Don'ts" of writing survey questions, and then we will look at some of the "Do's".

Once the brainstorming of ideas phase has been accomplished and you have a good idea what the scope of your research is, it's time to get down to putting question ideas on paper (or online!).

However creating good survey questions can turn into quite an art form, as we all struggle to word our items in such a way that our participants will understand what information we are asking for. Recognizing when you've accidentally designed a survey question that is worded in a non-objective way is the sign of a good researcher. Let's see if you can spot the poorly worded question from this group of sample survey questions:


Now that we've looked at some question items that shouldn't be put into surveys, next time we can define some ways we can create objective question items that will provide us with the results we need!

In a recent post, I described the differences between nonprobability and probability sampling methods in online survey designs.  Probability methods are preferred if at all possible, because they allow you to make generalizations from your electronic survey results to a larger population or target audience.  One kind of probability-based sampling technique is called systematic random sampling. 

To employ a systematic random sampling design for your online web survey, you first select a case at random from your exhaustive population list, and then select further cases at identical intervals, determined by how many people you want to sample in total.  If you wanted to sample ten people from a population list of 150, you would then choose every fifteenth person after selecting someone in the first 15 cases (to ensure you will select 10 people in total).

This provides an easy way to obtain a random sample of your population list or sampling frame, because as long as your data is ordered randomly, you can begin simply by selecting any record or case and go from there.  This is an important caveat though: your records must be randomly ordered for a systematic sample to be effective. 

Take this example of survey sampling, let’s say you have a sampling frame (list) of people that is currently ordered alphabetically by last name, and you are interested in subgroup analyses by ethnicity.  It would be wise in this case to rearrange the records into a truly random order (i.e., not alphabetically), because last names from certain backgrounds may be more likely to begin with a particular letter.  While systematic sampling provides an easy way to generate a random sample for online surveys, you do need to be sure there is no hidden order within your population list or sampling frame.

Some people still believe in using paper surveys instead of designing online questionnaires, sending personalized email survey reminders and utilizing electronic survey methods. Sometimes I have to wonder, Why? Some of the reasons people claim they want to use paper questionnaires instead of web survey forms is because they think it's cheaper. That's not usually the case.

Consider this: Online survey response rates are typically hire than response rates on a direct mail survey. Online survey research designed well will yeild response rates from 20-40%. Let's be conservative and say the paper survey you plan to send out will get a 20% response rate as well. I looked up the cost of a ream of standard multipurpose paper (500 sheets) on Office Depot's website: $7.09. A black ink cartridge for a laser printer is $77.99 and will print on a maximum of 2,200 pages. Most printed surveys will be kept to three to five pages, longer than that your survey form looks over whelming from the very beginning and is going to lower your response rate. This is how that cost quickly adds up:


Keep in mind these costs don't include the staff time of stuffing envelopes to send out the direct mail survey, or the staff time of manually entering in all of the responses into some sort of data collection software, be it Excel or a survey software tool. You can figure it will take an intern a week to send out 5,000 survey responses and another week for them to do the data entry for the 1,000 responses. So that's an additional cost of $600 to $800 for one survey campaign. What if you want to do a quarterly customer satisfaction survey? You're looking at an annual cost of over $25,000.

For me, that would be enough reason alone to look into an online survey software application. With a web survey tool, you have the ability to not only send one invitation, but reminders survey emails as well. Sending reminders, as we've mentioned before, can drastically increase your survey response rate. My suggestion: don't fall for the myth that paper surveys are cheaper than online surveys. It's definitely the other way around.

There advantages and disadvantages to conducting web or electronic surveys as opposed to traditional survey modes such as personal interviewing, telephone and mail.  One of the advantages of an online survey design is a possible reduction of what is known as social desirability bias. 

Social desirability bias occurs when survey respondents offer responses that portray them in a positive or more favorable manner to others. 

When a face-to-face interviewer asks personal or sensitive questions respondents feel have a “good answer” and a “bad answer” (such as criminal behavior), they may underreport bad behavior and overreport good behavior, for example.  While this bias may be reduced in online surveys due to the absence of an interviewer, there are topics that may produce invalid or unreliable data, regardless of survey mode.  This is something to consider before finalizing your online questionnaire design or web survey forms.

Survey questions within the following content areas are especially subject to social desirability in a survey form (i.e., respondents believe particular responses are “better” than others). 

The benefits of online and email surveys are numerous. This is not to discount other survey methods but merely to point out the advantages.

1. Cost savings. It is fairly inexpensive to email survey questionnaires, once you have the email addresses of your target audience.

2. Ability to attach pictures or sound files. You can attach or embed necessary files in the online web survey – you cannot show someone a picture over the phone, or send an audio file easily through direct mail.

3. More honest responses. It’s been shown that respondents are more likely to reveal truthful information online than to an in-person interviewer. Think about how much people reveal through personal blogs and web sites, and this makes sense.

4. Limitless possibilities. With certain kinds of survey software, you can control what types of questions survey respondent can access, show additional questions if they answer a certain question in a particular way, and more. This can be done with paper surveys but it’s a bit more cumbersome and requires written directions: If you answered 'No' to the above question, please explain.

5. Fast results. As I discussed yesterday, the results of online surveys are very rapid, which saves researchers time and money.

Technology has come a long way – ten years ago, the majority of surveys were not online, and that has certainly changed. Now market researchers are using this format more often than not for at least one aspect of gathering feedback. Whether you're looking to conduct a product enhancement survey or an internal business survey of employees, web survey forms have their place in your internet research process.

A key marker of a quality survey, whether that survey is an online form or another mode, is replicability of your results over trials. Replicability of your survey results lends credibility to your organization's research work.

One indication that your survey form may have results based on biased samples is nonreplicable results for the same instrument, fielded with the same survey design, from one time to the next.  If the same sample frame produces significantly different results for identical questions, that should perk up our noses to the scent of possibly unreliable results. We should start to look at the criteria used to select our survey sample, and take a fresh look at the questionnaire to see if it is still pertinent to our needs, or the needs of our clients.

This holds true for all modes of survey research, including online survey internet research. Receiving divergent findings in survey results being fielded again and again could lead to differing (and possibly harmful) business decisions.  No company should be left with uncertainty about answers and actions when decisions need to be made based upon the research. This is just one reason why replicability of your research findings is a key ingredient to sound data. When in doubt- replicate!

What is a survey design's ideal length? In the past we've told you the typical survey best practice says surveys, whether they're paper or web survey forms, shouldn't be longer than thirty questions. While thirty certainly isn't a huge number, if you tell me you have a thirty question survey you want me to complete, forget it. I'm never going to complete your feedback form. However, if you tell me it's only a 10 question survey, I'm more likely to be on board. That is, unless I start to give you the customer feedback you're looking for and feel like your online questionnaire is longer than 10 questions.

You probably didn't lie to me, at least not if you're read about the dangers of lying about survey length in your email survey messages. It's probably a case of side view mirror syndrome: objects are closer than they appear - or applied to web survey design, questionnaires appear longer than they are.

Not surprising, if your survey form is interesting, the survey respondent is going to perceive it's shorter than it really is. However, the reverse doesn't seem to be true. If your web survey feedback form is boring and on an uninteresting topic, respondents will perceive it's the same length (and too long) whether there are fifteen questions or thirty questions.

It's not just about the actual length of your form when designing a questionnaire online, it's also about the perceived length. Even though your survey is only ten questions long, it may feel a lot longer if your collecting feedback about an uninteresting topic. If your survey questionnaire feels long, you instantly increase your chance of survey respondents abandoning your feedback form.

There are many options available to market researchers when designing a web survey form. I covered this topic a bit last week when I discussed creating online surveys, and included some dos and don’ts.

Conducting a study for the Web offers several opportunities for market researchers that paper questionnaires do not. For example, you can create an instant survey and check on responses frequently for updates. You may even allow the respondents to view the current results when they complete the survey. This works best for one or two-question quick polls rather than longer studies. But knowing they will see the results immediately is frequently a tipping point for respondents who are on the fence about completing your feedback survey form.

This can be helpful in analyzing survey data as well. For example, suppose results for your online survey form vary wildly throughout one week. One day, 10% of respondents use your client’s brand of glass cleaner. The next day, it’s 65%, but then the following day it drops back down to 20%. This could mean something (possibly that you need to check your recipient list) and is good information to track for your client.

The instant results also allow you to add more questions and gain clarification on points you may not have considered prior to launching the survey form. There are dozens of survey web software tools available that can create instant survey results – it’s a method that is worth trying out, but as always, consider your client’s needs before committing to a particular survey method.