Survey Research

There are literally dozens of ways respondents can misread and even misconstrue survey questions.  Some potential errors may be easily identified, while other errors can go unnoticed all the way up until data collection begins.  The possibility of a large number of respondents skipping the same questions, or customers providing invalid feedback because of faulty survey questionnaire design and implementation, are just two of the many reasons you should test-run a survey before sending out the real thing.  Conducting a pretest is the single best way to identify phrases subject to misinterpretations (and question design flaws in general).

Pretests (also referred to as pilot tests) are used to test the validity and reliability of individual survey questions, the entire questionnaire, and/or response scales.  While carrying out a pretest adds a step to your task list, the benefits of sending out a valid instrument far outweigh the costs of a pretest (which for online surveys usually means only extra time, not dollars).

To conduct a pretest, you first need to select a smaller survey sample that is still fairly representative of your target population.  You then have this pretest sample complete their surveys, while providing feedback about your questions, any technical concerns and other elements you feel could introduce bias or error into the process and subsequent results.  These respondents will no longer be eligible for actual survey participation since they will have been exposed to the questions ahead of time, but the process is well worth it.  The final step in the pretest is to analyze your results and decide how you should revise the survey to make it better and easier to complete.

One big caveat: make sure you use people unfamiliar with your research.  If you use colleagues or acquaintances who are knowledgeable about any relevant topics or technical issues, they may overlook correctable errors such as leading and loaded questions, faulty transitions and overly technical terminology.

By taking the extra step of conducting a pretest, you can gain valuable information from a small subset of people.  This will allow you to make the necessary and appropriate revisions, and in turn produce valid, reliable survey questionnaires for your formal survey research project.

When working with a client who has already built a strong customer base, market researchers are often called upon to create surveys to aid with customer retention programs.

Customer retention surveys are a bit different than a new product or service launch survey because the survey respondent sample must consist of product users. There are several things you need to ask this group so your client can formulate the best customer retention strategies, including:


The recommendation sample customer questionnaire question is important. Often, this can be a company’s bread-and-butter when it comes to retaining customers and gaining new ones. If a good amount of current customers would recommend the product to a friend, that speaks volumes about the quality.

It is also wise to include some open-ended questions. Find out why people use the product, and would continue to do so. If they wouldn’t recommend the product, or use it again, it is important to know why. Your client can use the information to persuade customers on the fence, and keep current customers coming back.

Before creating a poll online for your client survey, sit down and gauge initial opinions and move forward from there. If there were original product launch studies done, it may be helpful to revisit those results as well, just for comparison purposes. Approach the questionnaire research with care and the resulting data analysis will provide information to create customer retention techniques and marketing strategies.

If you follow the bloggers on the Cvent Web Survey blog, you know we emphasize survey reporting concerns throughout the entire questionnaire project life-cycle.  This post outlines five general sections that comprise a good report or presentation for a survey research project.  By following this outline, you can be sure that you have a thorough summary of your research objectives, processes and results all in one place at the conclusion of your survey project.

Executive Summary
This is the "drive-by" version of the entire survey project.  It contains a brief summary of your main points, and predominately consists of broad research findings.  You can also think of this as the audience takeaway section: imagine if people had just five minutes to hear your results, and then had to make decisions based upon what they heard.  Include essential, bottom-line insights from your survey project here (and not much else).

Introduction/Background
In this section, you introduce your study, including its purpose, concepts, rationale and variable descriptions (where appropriate).  This is also the place to provide all the background information, as well as any prior research findings/citations upon which your current study is based.  If you have specific hypotheses, they can be placed here after you’ve established rationale.

Methodology
This is the part of the report where you get dirty with the details of how you went about designing and conducting your survey.  Descriptions of survey sampling procedures, survey question and response option/scale design, data collection activities and timelines all belong here.

Results
In the survey results section of your report or presentation, you provide all the survey findings.  This includes detailed tables, charts and other selected graphics that clearly explain what the data say, and can be quite extensive depending on how many survey questions (and research questions) you have.

Conclusion/Discussion/Recommendations
These sections can of course be listed separately, but I’ll group them together here for our general descriptive purposes.  This is where you take the step from findings to decisions.  You’ve detailed your results; now you draw conclusions, discuss anything that might have affected the research or your path forward, and make recommendations about where to go next based on the data.

Cvent’s online survey tool is like a treasure trove.  Every day, there are new features to discover, new ways to utilize different functionalities and, of course, a never ending feeling of adventure as you navigate your way through.  With all the excitement, it is easy to overlook some features or get stuck in a routine where you use the same survey question types, same images, etc.  for every survey.  Regardless of the purpose or goal of the internet research survey, you will find a way to make those features you are comfortable with work.  But, come on now.  What kind of Customer Support would we be if we didn’t educate you on features that may be underutilized but have enormous potential to change the way you look at surveys forever?

Cvent offers 19 different question types; that is 19 unique ways to ask a question and 15 unique ways to collect a respondent’s rejoinder (single and multiple select questions have 3 different formats each).  My recent favorite, which I have seen in very few surveys (and there are only so many formatting ways to stress VERY), is the File Upload question.  This feature allows respondents to upload additional documentation which will be stored in one, central location.  Think about the possibilities:

1. With the amount of free and easily accessible information available on the internet, online tests have become increasingly difficult to administer.  What is to stop a student or anyone for that matter from "just having a peek" at Wikipedia or "accidentally" having a Google page up with the pertinent information?  A study done by Donald McCabe and Linda Trevino found that schools that had an enforced honor code have significantly less prevalence of cheating (McCabe, D.L. & L.K. Trevino, 1993). How can this be accomplished through an online survey?  Have respondents sign your school’s/organization’s honor code then use the File Upload question to have them upload that document. 

2. When I mentioned that Cvent’s web based survey tool can do anything; I meant it.  You can use the survey tool to identify and confirm respondents eligible for rebates offered on a purchase.  I know personally I get a little perturbed when I see that an item costs $X but, when I get to the register, I find out that it actually costs $Y but I can get the appropriate rebate in 4-6 weeks if I just mail my receipt to the company.  Call me impatient, call me lazy, but I don’t like having to go through all of that.  “But, Dorian, how can we do this differently?” you must be asking...  Two words: File. Upload.  Customers can upload the receipt from their purchase which will be stored within your Cvent account instantly.  If your survey is a collect contact information survey, you can ask for their address and the only step would be to verify the receipt/purchase and then send out that rebate.

3. Respondents can upload images or graphics; photos for a high school reunion, anyone?

The possibilities are endless.  Any additional documentation can be collected and you don’t need to overwhelm your inbox with email attachments for each respondent.  So go forth, I say!  See what other survey treasures you can find, Captain Sparrow.

Before you select a sample or design a single question, the first step in a survey research project is to establish your goals and objectives.  With proper planning, time and attention given to this step, all subsequent steps will flow more smoothly. 

Ask yourself--and answer, in writing--broad questions such as the ones below, and then you can drill down into more specific research questions and move on to procedural matters.

What information am I seeking to gather?  Why do I need it?  What is the “end game” regarding what I’m trying to learn?

Who (exactly) can I get this information from?  How can I contact them?  How many people do I need to contact, and do I have an exhaustive list?  Am I interested in any particular subgroups?

Has anyone else done a similar project in the past?  If so, what did they find out and how could what they did help with my current project?

What kind of data and results do I want my questions to produce?  Am I interested in public opinion, actual behaviors, satisfaction levels, employee feedback/workplace surveys, etc.?  How will I go about collecting and analyzing the data?

How will my results be reported?   What will the report format look like, and who will receive the results?  What kind of summary data do I need, and how can I design questions with response options that will provide me with actionable data?

Establishing clear, attainable objectives and goals is of the utmost importance when undertaking a research study.  Your results will likely only be as clear as your objectives, for better or worse!

I was asked to complete a product market research survey today. The product survey asked me a series of questions about possible products related to improving my beer drinking experience. One of the many things I didn't like about this survey was they never asked me if I thought there was a problem.

For example, one of the possible products was for a different method of carrying beer. Unfortunately, I'm perfectly happy with my current method: keeping the beer in the package it comes in. The suggested product would make me move the bottles from their packaging to take them somewhere. I'm not interested in that extra step. Did they ask me why I wouldn't by the product? Nope. They only asked if I would.

Maybe for their survey research goals, that's all they cared about: Would I buy the product, Yes or No? Kelli mentioned in a post earlier today that it's important to gather product feedback on a smaller scale before launching a bigger market research study. The goal of this market research questionnaire easily could have been to identify which product ideas should be considered for further qualitative and quantitative research. Only products where more than 20% of survey respondents indicated they would be interested in purchasing will have more research around it.

On the other hand, I like to make sure my position is clear when completing attitude surveys. This is probably a respondent bias I bring to the table because I spend so much time sharing online survey best practices. I know that often survey questions are written in a way that doesn't always collect the data the survey creator wants. They just don't always write the right questions. So avoid being "mis-quoted" so to speak, I like to be able to answer a series of question about why I feel one way or another.

Believe me, there were a lot of poorly written survey questions in the feedback form, as well as other issues with their data collection method. I've found that where there is one serious problem with a survey template, there are others hidden as well. If you're worried your survey questionnaire may have problems, go ahead and ask our online survey Professional Services Group to take a look. We're always happy to help.

Looking for sample market research survey questions? Here are a few sample product survey questions you can use in your next product research survey:

Private businesses tend to guard findings from their research efforts very closely.  In some instances, where significant investment dollars and time were spent on sampling frames and reporting, this makes sense.  It's nice to have proprietary research at your disposal to impress clients and customers, and internal employee survey results, for example, may be reserved for senior leaders.

But in most cases, I suggest sharing your survey results with as many people as possible.  The benefits outweigh the perceived drawbacks.  After all, most survey research comes from of a specific point-in-time sample, and whatever narrow edge you may get from keeping the results private will be short-lived.

At minimum, you should share the results of your survey with respondents themselves.  People like to know what they're a part of (and why).  In fact, offering to provide respondents with survey results has been shown to increase response rates! 

It's a great idea to set up partnerships with community leaders before you even send out your community attitude survey.  Publicize your organization and your efforts toward being proactive.  Let your community know that you value them and are interested in feedback and suggestions about the process.  Create anticipation and eagerness to both complete questionnaires and receive the results after the survey is completed.

You can also organize a press release highlighting selected findings from your survey.  This can be posted on your website and in other locations (external websites, community hotspots, etc.).   Call local news organizations if you think the results would appeal to their audiences.  Tell all who will listen how you've made original contributions to the knowledge of your industry, and how you will improve business practices based on your results.

Market research is incredibly useful at any stage of the game. Though starting out small may seem like an unnecessary step, sometimes it works out better for you and your client. A thorough study is necessary for any new product or service launch. When dealing with a NEW concept, you may want to conduct a smaller initial study to gauge interest level before continuing.

When conducting the smaller study, set criteria for continuation of the study. Criteria for every client and every study will be different, but the goal of setting parameters should always be the same. Decide what percentage of respondents must be interested in order for you to further develop the concept before continuing the study.

For example, if you gathering product feedback for a major cleaning product company about a brand-new product, you might want 60% of product market research survey respondents to express an interest in order to develop the study further. A full product market research study is extremely time-consuming - the entire project will likely include multiple online product surveys, focus groups and other data-gathering methods.

Market research projects like this have a lot of moving parts - your team will have to write survey questions, compile results and create presentations based on their market research analysis of the data. If it turns out that only 20% of respondents are interested in the new product, it may be better to return to the development phase before continuing with the study.

Even though it might be tempting to forge ahead, particularly in lean economic times, your clients will appreciate you more if you save them time, effort and money by starting out small.

Most survey poll creators will agree that while respondents always prefer to have choices explained clearly in words and phrases, numbers are much easier and far more effective when drawing inferences from the data collected. Take this typical sample customer service satisfaction survey question:


The client satisfaction survey sample question above from a measuring customer service sample survey looks much better by listing the choices as "Very Satisfied/Dissatisfied" as opposed to asking respondents to check one of the numbers between 1-5.


However, when the survey data collection is complete and it's time for data analysis and running survey reports, many survey administrators are left scratching their heads. What analysis do they draw from the individual number and percentage of respondents listed as "Neither Satisfied nor Dissatisfied", how to they compare these percentages to last year’s survey on a category by category basis and still draw an overall inference? How do they know the percentage increase in the level of satisfaction? Is it better to have a large number of people fall into the neutral category?

The answer to all these questions is the same – Export Values. The ultimate tool for anyone looking to analyze the data they have collected. Using Export Values the survey builder can assign a number (or a different word or phrase if you like) to answers as well as questions in their survey! In other words, instead of going through the never-ending text of questions and answers trying to find some semblance of meaning in the data, the questions in your report could be listed as CS1, CS2, CS3, etc. and the answers as 1,2,3,4 and 5 for each question. After exporting the Expanded Answer Details for All Respondents (the survey report with Export Values) to Excel, you can even use these numbers to calculate a mean or average – how much easier would it be for the decision makers in your organization to see the response to the above question summarized into a single number:


A case in point is our survey clients who go a step further and used this feature to export just such a report to Excel and then analyze it using SPSS and other survey research analysis software. For those of you who are already drooling over the prospect of generating such accurate feedback for your organization, export values are pre-included in your Cvent Web Surveys Professional or Enterprise account. Go on, generate that perfect report – you can thank me later.

A recent article in Meetings & Conventions Magazine caught my attention this month: Survey Science, How to Craft more Effective Attendee Evaluations. Meeting planners are usually not survey experts, their expertise lay in planning and executing events, not crafting flawless survey research questionnaires. However, in the last year the value of meetings and events have been called into question after bad press surrounding some organizations' meeting practices. With meeting and event planners having to increasingly prove the values of meetings and measuring meeting results, being able to create event surveys to gather attendee feedback has become even more important.

While we've discussed several of these survey best practices in the past in reference to sample staff survey questions or customer feedback forms it never hurts to review them again. Here are the first five online survey tips from the article:

1. Set survey goals first. This shouldn't be a big surprise. On Friday I wrote about how to create a customer survey, and setting customer satisfaction survey objectives was the first step there too. If you don't figure out what your objectives are in the beginning, writing meaningful survey questions will be a lot harder later on. For example, if the survey objective is to figure out how to improve the event next year, only ask questions about things you can change for future events. It doesn't make sense to evaluate the location if you've already booked the same venue for next year; if you haven't booked next year's location, asking about the venue makes sense.

2. Write survey questions that are clear and ask what you really want to ask. Articulating a question to really collect the feedback you're looking for can be quite a challenge. Take these sample conference survey questions, if the planner wants to identify which sessions would be best to add to next year's lineup (assuming attendees must pay to attend the sessions):

If you've been paying attention to survey best practices, you'll know the third one is the wording you want. Why? Because it's the only one that explains the attendees will have to pay for the sessions. It's not a matter of which sessions sound interesting, it's a matter of which sessions present enough value to be worth the cost.

3. Find out why they're dissatisfied. It's not enough to know that an attendee was dissatisfied with the registration process. Planners should provide attendees with the chance to explain why they are dissatisfied - maybe it was a technical problem or there wasn't enough event staff to mitigate issues.

4. Keep it short. General survey best practices suggest keeping your online survey as short as possible. The best methods for making sure you're keeping it short is to only ask questions that have a direct link to your seminar survey objectives. The second method is to make sure you're using survey question logic, such as branch and skip logic. This allows you to keep the survey relevant to the respondent and only ask questions that make sense. We hinted at this in a recent post where we talked about using one survey to gather event feedback for all the sessions at your event.

5. Think about the survey's organization. When designing questionnaires it's important to keep the flow of the survey in mind. Start with general questions and work your way to the more specific questions. The survey should only focus on one topic at a time. In the example of multiple event sessions, it doesn't make sense to mix up the session questions. Instead, ask all the questions you have about one session, then move on to the next session.

In the next few days, I'll share the other 10 survey tips from the Meetings & Conventions article. As a meeting or event planner, how have you found pre- and post-event surveys helpful in planning events?

Have a question? We're here to help! Here at Cvent we're always trying to help people improve their survey projects. Whether you're trying to write new employee feedback survey questions, create a customer service performance reviews or simply get a better understanding of how our online survey research software and enterprise feedback management solution works, we're happy to answer your questions!

To make it easy to get all your survey questions answered, we've added a chat to our blog. Someone will be happy to answer your questions during normal business hours (9am - 6pm ET). So go ahead, click to chat and ask our experts your questions about employee feedback software, data collection methods, customer survey questions, or any other survey questionnaire related question!

Many people ask the question How do I create a customer survey? The basic steps are the same no matter what type of client survey you're writing: customer satisfaction, global market research, product development surveys, etc.

Step 1: Sit down and figure out what the goal of the survey is. Are you trying to identify upsell opportunities? Want to discover features missing from your current product? Figuring out if it's a good idea to take your marketing overseas and attack a global market? In the very beginning of the survey planning process, you should know what it is you want to get out of the consumer survey. If you don't have a firm customer satisfaction survey objectives in the beginning, while you go through the other steps such as writing survey questions or selecting the best survey software, you're going to stray from the path. If you stray from the path, you may find the final survey results are not as helpful as you had hoped.

Step 2: Decide on a research methodology. Your goals should help you on this step as well. You need to first decide if you're planning to do qualitative or quantitative research. From there narrow the scope further, if you want to do qualitative research are you interested in focus groups, advisory boards, one-on-one interviews? With quantitative research you may decide on comment cards, feedback forms and surveys. Is your survey method going to be online, telephone-based or paper questionnaires?

Step 3: Survey Design. I'm making the assumption since you're reading a survey blog about how to create customer surveys, you're not interested in the other market research methods right now so I'm going to focus on the process of building customer surveys. Once you've gotten through the first two steps, you're ready to start writing survey questions (Finally! I bet you thought this would be the first step!). Customer satisfaction survey design can be the biggest challenge. Luckily, there is survey designing software to help you step through this. Survey software tools often have templates and question libraries to help you write good survey questions.

Step 4: Data collection. Okay, you've picked your customer survey methodology, created a client survey and you're ready to field your survey (or use the data collection tool in your survey application to collect responses). Exactly what you do in this step will depend on what type of survey you decided to collected: telephone, paper, online. One way to get survey responses is to use email marketing tools to send personalized email surveys. You can also share the link on your website, social media sites, invoices, etc.

Step 5: Analyze customer feedback. Analyzing survey data is one of people's least favorite parts of the surveying process. We have some tips for how to analyze survey data here. Don't be afraid of this step. You need to conduct the survey customer analysis to achieve your goal. It's what you set out to do, so keep your chin up. You're only a step away from the final product (and once you choose survey analysis methods you should be almost finished).

Step 6: Share the survey findings. This is what you set out to do. Get answers to your customer questions. Take the customer feedback analysis you completed in the last step and format it. You're creating a survey report you can share within your organization (and maybe with others outside of your organization). If you need tips for creating survey reports or an example survey report, you can read more about them here.

Step 6.1: Take action. This is still part of step 6, but it's important enough it should be broken out. In your customer analysis survey report, you should have shared your recommendations for moving forward. Make sure you make recommendations and there is an agreement about moving forward based on the customer survey findings. If you don't plan to take action in Step 1, then you should save yourself the time of conducting the customer research in the first place.

Any other survey research design tips? How have you used these steps to create a customer survey that improved processes in your organization?

Although question order effects are believed to be stronger for interviews than for online research surveys, it is still important to take a look at your market survey format and how the questions are ordered throughout your survey questionnaire.  By following the general guidelines listed below, you can reduce possible order effects and response bias from your customer survey samples.

Start broad, then get more specific.  Ask your customers or respondents general questions about your organization and the concepts you are interested in first, then get down to the more detailed questions you want them to answer (such as those about individual products, specific preferences and how to improve customer service).

Ideally, you’d like to find out if the order of your questions can induce biases before you send it to your entire survey population.  Use online survey software that allows you to randomize parts of your online survey, and compare the results with those from predefined orders.  This is an excellent way to pretest your survey questionnaire for possible order effects.

Devote some time and effort to making your survey “flow.”  You want the overall survey instrument to be cohesive, not disjointed or seemingly all over the place.  A big part of allowing for a logical flow throughout a survey comes from writing smooth transitions.  Let your respondents know that you want them to change gears.

Ask demographic questions at the end of your survey.

If you are asking sensitive questions (about topics like income, personal behaviors, etc.) try to place these toward the latter parts of your survey as well.  That way you get at least partial completions in the event that respondents change their mind and exit your survey before full completion.

As is the case with validity, there are multiple aspects of reliability in survey research.  In general though, while validity refers to accuracy in question design, reliability refers to the consistency of your results (i.e. the probability of producing the same results after repeated measurements). 

One way to easily conceptualize reliability is to consider it as another word for precision.  Let’s use archery as an example: if you shoot ten arrows at the target and they all hit the same area, your shots can be said to be reliable or precise. 

You can test for and estimate the reliability of your survey questions in a variety of ways.  You could use a test-retest design, where you use the same questionnaire more than once to see if the results remain consistent.  You could also send your feedback form to different survey samples (consisting of similar respondents) and then compare your results for any fluctuations.  Also, if you create a set of customer service questions designed to provide you with a composite scale, you would want the individual questions to produce similar results if they are intended to measure a single concept (an overall customer service scale, for example). 

It is also important to understand the relationship between reliability and validity; results can be reliable without being valid.  If we use our archery example, the shots are reliable, but if they aren’t near the bullseye (an undesirable outcome!) they cannot be said to be valid.  Reliability is necessary for validity, but it is not sufficient alone.  You want to accomplish both objectives: shots that land near the bullseye consistently.  By creating reliable, valid questions for attitude surveys, online market research, employee reviews or public opinion polls, you can be confident that your results are not due to chance.

When a product fails, there could be many reasons – poor economy, too much competition, or a lack of market demand. Famous examples include “New Coke,” Crystal Pepsi or Sony Betamax.

These products all failed hugely and publicly. While the advertising was blamed in many cases, it’s also true that at least part of the fault lies with poor market research. If proper focus groups and quantitative research surveys had been conducted for any of these products, the embarrassment and monetary loss for all three companies could have been avoided.

In focus groups for New Coke in the '80s, the samples of New Coke were well-received. However, participants were also satisfied with current Coca-Cola offerings, and some had strongly negative reactions when asked if they would buy this new soft drink if it replaced Coke Classic. The survey was less negative, but if the focus group results had been better used to create the quantitative research questions, things could have gone differently. 

For example, the consumer survey could have contained questions such as:

1. Are you satisfied with the original Coke formula?
2. Do you regularly purchase Coke products?
3. Would you consider buying a “New Coke” product if the flavor was different?
4. Would you buy the “New Coke” if the new formula replaced Coke Classic?

These types of questions might have given a more well-rounded view of public sentiment, like the response that happened in the focus groups. Although it might seem less exciting to have a negative result to a marketing research survey, overall the company will save money, and avoid negative press. Paying for a study is cheaper than launching a product no one wants, after all.

A new public opinion poll was commissioned by SCHOTT Solar and Solar Energy Industries Association (SEIA) for the annual SCHOTT Solar Barometer Survey. The attitude survey found 92% of Americans think developing and using solar energy is important for the United States. With all the political debates going on in Washington, this does not rank among them. Survey responses were consistent across all political affiliations and ideologies. 77% of survey respondents indicated they think the development of renewable energy sources should be a major priority for the federal government, this includes providing financial support when needed. The most favored renewable energy source is solar (44%), followed by wind (17%), natural gas (12%) and nuclear (10%).

The private sector sees the possibilities in the solar space as well. They aren't waiting for a web survey to tell them. According to Cleantech Group, the "cleantech sector" has grown almost 25% over the last 5 years. Just in the last quarter, solar was the leading sector. There is still plenty of room for growth in this area though. The SCHOTT survey found that almost half (49%) of the survey sample are currently thinking about solar power options for their home or business. This is a big opportunity for organizations looking to develop solar energy solutions.

One think I always tell you to look for is the survey methodology used to collect the data and draw conclusions. According to the SEIA press release, a third party market research firm conducted the online survey between August 31 - September 8, 2009. Email survey invitations were used to solicit survey responses to the online market research survey. Due to the quantitative market research method, the survey results are 95% accurate within plus or minus 3.1 percentage points.

You want your employee performance surveys, public opinion polls, and online market survey studies to produce valid data.  But what does validity mean exactly in the field of survey research?  Although there are actually many different types of validity, in general, designing valid survey instruments means that survey questions adequately measure (in reality) the concepts the researcher is attempting to measure.  As opposed to reliability, which refers to producing the same results with repeated measurements, validity involves designing legitimate, suitable survey questions that accurately address the ideas you’re seeking to evaluate. 

Survey researchers try to attain high levels of external and internal validity. External validity is the degree to which your study’s results are generalizable to individuals other than those in your survey sample.  If your results are externally valid, you can come to increasingly accurate conclusions regarding larger and different populations (people/markets across various time periods, geographical locations, etc.).  Obtaining external validity is a major goal of survey and market researchers, because most research is performed using a small sample of people from a larger population or target market in order to decrease costs and lessen workloads.

Internal validity, on the other hand, centers around estimating how valid your measures are based on an inward look at the research design and any other variables that might interfere with accurate results and conclusions.  To increase internal validity, survey researchers often like to implement experimental designs, as well as perform statistical analyses that control for variable effects.  Examining any additional measures and proposing alternative explanations for results both go a long way in judging the validity of any survey research project.

According to an Ad Age article published today, there is no longer an average American. This doesn't come from a consumer survey they conducted, but from the expected results of the 2010 US Census. There are approximately 309 million people living in the United States, and over time, we have become a complex, multidimensional society.

For consumer marketers, this may be a big shift. No more "the American consumer," no more "Average Joe." The U.S. Census is the largest market research project of the decade. The Census Bureau will be gathering data on American consumers and spending upwards of $15 billion doing it. There's a reason that Peter Franceses, demographic trends analyst at WPP's Ogilvy & Mather, New York, calls it the gold-standard of consumer market research studies.

Here are some of the expected findings Franceses shared with Ad Age:


What does this mean for consumer marketers and survey samples? Getting a representative survey sample is becoming even more important. With the death of the "Average American Consumer," market research survey designers can no longer rely on data collected from just one group. When making business decisions, organizations will need to look at the entire picture.

What else? Focusing on niche markets will become more important. It's already very difficult to please everyone, but it's getting harder and harder. Consumer products that will work for a single couple living in New England is not necessarily the same consumer product that will work for a blended family in California. It's important to make sure you're conducting your own product market research. While the Census is collecting a lot of information about consumer trends and consumer behavior, organizations still have to conduct their own market research related to their particular markets.

Survey writers from various industries conduct surveys using the Cvent Web Surveys software tool. One of the most common industries is customer service, where the survey creator is trying to make sure their customers are happy with their products and services. Customer service surveys are used to ensure the quality standards of customers satisfaction.

When the Cvent clients conduct a customer satisfaction survey, they wait for the survey responses to be completed before running reports to see how respondents feel about their offerings. What about the respondents who were dissatisfied with the product or service and want to be contacted immediately?

Email alerts are one of the most useful features that survey designers can use to identify and then take action, where appropriate, when respondents say they are unsatisfied with the service or product. Email alerts provide the option for the person creating the survey to set alerts on questions, this can be a question where the survey respondents are asked about their satisfaction level with the organization's offering.

Take this example customer satisfaction question: "How satisfied are you with the quality of the product?” If emails alerts were set up, an email will go out to the specified people letting them know a customer selected the survey question option of “Dissatisfied."

When someone answers the question and says “Dissatisfied” an email will be sent to the people who were added to the email alert. In some cases, it may be the same person who created the survey, in others, it may be the sales person who owns the territory the respondent is located in or even the customer service manager. The employee or group of employees can quickly get in touch with the customer to inquire more about their experience. Why do organizations want to do this? Because closing the feedback loop can make the respondent feel valued and in some cases move them to the satisfied customer column.

The email alert function doesn't only work on the question level, you can also use email alerts on the survey level and receive an email anytime someone completes the online web questionnaire, or you can set email alerts based on respondent scores. If you're using scoring in your survey questionnaire, you can set alerts to be sent if a respondent falls above or below the specified score. While we used the example of customer satisfaction questionnaires to explain the value of email alerts, they can also be used with employment performance review forms or market survey questions or other types of web survey research.

In my last post I described how to use question language that is appropriate for your audience when designing customer market surveys or conducting online polling research.  This is a great way to ensure you obtain meaningful results and reduce one source of survey design error.  Two additional concepts- relevancy and competency- are factors in questionnaire design that have consequences for the validity of your results as well.  This two-part series will help you design survey questions that take into account both of these essentials.

Designing items that are relevant for your audience is necessary in order to write good survey questions.  This may seem self-explanatory, but many surveys I see are actually all over the place with regard to content.  Some companies send out “omnibus” surveys, which may ask what you ate for dinner yesterday and what you bought for Christmas last year in the same questionnaire.  An omnibus survey is one that covers a variety of topics, often because multiple groups are sharing the cost of the survey forcing the focus to be spread out.  It is better to focus a survey respondent’s attention on the relevant information you’re attempting to gather (usually one or two concepts). 

Stick to your theme!  It is tempting to include every question you can think of in a market survey or public opinion poll, but you may end up with market research survey results that are “a mile wide and an inch deep.”  Heaps of data that contain little insight can amount to wasted time and effort for all involved.  If you send out a survey with the word technology in the invitation and title, make sure you include questions that relate to the main concept throughout your survey.  You want usable data and actionable information; you can get both by keeping your focus (and helping respondents do the same).