Survey Research

Toyota fans have not wavered in light of all the recent Toyota recalls, according to a recent study undertaken by Rice University. These results don't only apply to Toyota owners, even non-owners seem to have maintained their levels of admiration. Customer loyalty has insulated Toyota from the negative press surrounding the 8 million plus recalled vehicles.

Rice University's Jones Graduate School of Business conducted a rigorous survey research study of vehicle owners. Using a national representative sample, the research found the level of overall satisfaction with Toyota vehicle quality was about equal between Toyota owners and non-owners. Not surprisingly, Toyota owners did prove to have a more favorable opinion of the brand than non-owners. Despite numerous recalls and bad press, owners seem to look at these events in quality and engineering issues as an anomaly.

What I found most interesting about this study's survey results was the fact that Toyota owners believe other manufacturers are just as lax on safety and domestic brands (GM, Ford and Chrysler) are not catching up to Toyota and Honda in either safety or reliability. On average, Toyota owners said they would consider Toyota if they were looking to purchase another car today (average rating of 8 on a 0-10 scale). On the other hand, non-owners gave an average rating of a 4 using the same scale. So it seems that while there is a brand insulation effect in play with the recalls, non-owners are not as loyal as Toyota customers.

This survey research was conducted between February 20 and March 2, 2010. Rice researchers surveyed 455 vehicle owners. 12.8% of the survey respondents were Toyota vehicle owners, which is representative of Toyota's current market share. Of those surveyed, 93% were aware of the recalls.

All too often, businesses fail to conduct one of the most important steps in marketing research: understanding what the consumer really thinks. For businesses operating on small budgets, market research may sound scary. However, finding the right real time online survey software can be the ideal answer to meeting market research needs while staying under budget.

Focus groups provide great insights into consumer attitudes, opinions and general thoughts, but organizing focus groups that deliver results you need can be very challenging - and costly. For example, if your clothing company is looking for success on a national scale, will the expenses of focus groups in New York, Los Angeles, Chicago, Seattle and Dallas provide the return you're looking for? Fortunately, web based surveys offer a great alternative. Conduct the customer research you need to better understand how consumers buy clothes at a fraction of the cost.

With internet research software tools - like an online survey application - you can collect the same data you would gather in person, but it's even easier to analyze. Rather than dig through hours of audio and video recordings of consumer feedback, conducting web based survey research can let you analyze respondent answers with real time reports, graphs and charts that help you better understand how to reach your target audience.

In a recent post, we learned about the importance of banking surveys to gauge consumer satisfaction levels during uncertain financial times. Another industry facing serious changes amidst a nationwide debate is the health care industry. For hospitals, doctor offices and other medical care facilities, health care satisfaction surveys are a great tool to measure an organization's performance while showing customers you care.

In addition to actual medical facilities using health care surveys, insurance companies can conduct survey research to review how well these places are doing. If a regional hospital receives lots of negative reviews in an insurance customer survey, the insurance company may want to rethink sending their policy holders there for care. Give patients a forum to voice their honest opinions about the issues that matter: how long they had to wait for emergency room treatment, how compassionate the staff was or if their bills were properly handled.

As the conversation over health care costs grows bigger and more confusing, it's crucial to understand how past and present patients actually feel about the care they receive.

Looking for an even easier method to gauge how well your organization is performing? With survey scoring, you can assign numerical responses to each answer to deliver an accurate picture of customer satisfaction.

For this customer satisfaction example, we will consider an athletic facility conducting survey research to collect customer satisfaction feedback. When the web survey builders begin to draft questions, they should base all questions on the same scale. Let's use a 5-point scale in this customer questionnaire creation example, with 5 being the greatest degree of customer satisfaction and 1 being the least. The survey creator can assign a system of point values to each question.


Survey Scoring Scale Example
With this scale in mind, let's consider some customer survey sample questions for the athletic facility:


Customer Satisfaction Survey Sample


Customer Satisfaction Survey Sample


Customer Satisfaction Survey Sample
Using these questions from a customer satisfaction survey form template, you can see how the scoring average can help simplify analyzing survey data. A respondent that scores between 12 - 15 is happy with the facility as it stands. Any respondent that scores below a 9 is unhappy with one or more of these areas.

You may also want to consider using a side-by-side matrix question to compare how satisfied someone is to how important the quality is to their over all satisfaction. Someone my feel the staff is willing to help, but it doesn't matter to them, however they've very dissatisfied with the cleanliness of the locker rooms and that attribute is very important to them. 


Example Customer Satisfaction Survey Question
When you create your next web based survey, consider using the scoring feature in your online survey system to make data analysis easy and effective.

Vignettes have been used in survey research for years, and in a variety of settings and with many different populations. A 'vignette' can be more commonly termed as a hypothetical story or situation inserted into a survey, and it is often used to elicit information about values, beliefs and perceived societal norms from participants.  Respondents react to what they read (or see if it is a video insertion) see happening in the story, rather than answering a vague question about, "in general, do you feel..."  More recently, video vignettes have been incorporated into web surveys.

Besides having a specific instance to comment on, if used properly, utilizing vignettes in your survey work can also allow you to ask tougher than normal survey questions. For instance, items about perceived racial or class bias, in an nonthreatening format.

This technique is very common in surveys of health providers, like medical doctors and nurses, because stories are a common teaching tool used in medical training settings. So for example when surveying a population of chiropractors or dentists, opening up your survey ideas to the use of a vignette might be a good idea, since those populations are accustomed to that approach through their training.

Incorporating vignettes can be a little tricky. It's important to keep the length of the story as brief as possible so you don't loose the attention of your respondents. After all, they aren't taking the GRE's!

However, it is also important to include as much detail into the vignette as possible, so respondents are able to answer your dependent questions. Asking no more than 5 follow-up questions for one vignette is ideal.

For more on the use of vignettes in surveys go to Harvard University's research on the topic.

In our last discussion, we learned about why surveying students is important to colleges and universities. To some degree universities are similar to businesses, keeping students happy is just as important as quality customer service.

Using the Likert Scale to Collect Feedback
While you can create educational survey questions in any form, using the Likert scale offers a very productive way of gathering data for efficient analysis. Below are three course evaluation survey template questions. Conducting academic survey research helps schools get honest and accurate feedback of how students perceive their educational experience.


Education Survey Sample Question


Education Survey Sample Question


Education Survey Sample Question
While the Likert scale helps average survey results into reports for easy analysis, it's also important to let students respond freely, too. Giving students an opportunity to provide additional feedback about any other thoughts, concerns and issues will help provide a full view of their opinion. In addition to questions based on numerical scales, just insert a blank text box in the web survey for these comments.

In Learning About Learning, we discussed the significance of conducting survey research in the educational world. Combing this lesson with an earlier post about understanding the Net Promoter Score reveals an important truth to university administrators: your students are your promoters.

Because a college's existence depends on students continuing to enroll, making sure that current students are happy will increase positive reviews and recommendations to potential students. Think about your college admissions department - they need glowing reviews from current students to help prospective students see the benefits of choosing their institution.

With this lesson in mind, many colleges are beginning to use web surveys for post-course evaluations. Students appreciate a chance to offer anonymous feedback about a professor's poor performance, a curriculum's shortcomings or any other issues they had during the semester. Remember that crucial statistic: most dissatisfied customers will eventually tell nine other people about their problem. That translates to nine potential students who may choose to go elsewhere for their education.

When you create educational survey questions, it's important to ask the tough questions. In our next post, we'll discuss how to create an effective education survey using the Likert Scale.

In the online world, appearance can mean everything. We've discussed making your email invitation stand out in an earlier post, but giving your survey a dynamic appearance is just as crucial. When you conduct survey research, one of the keys to getting a high number of respondents is making your survey look professional.

Templates
When you're choosing a web survey company for your data collection needs, it's important to know what kind of templates their survey software includes. At Cvent, the survey template library is extensive, featuring over 50 graphic templates that make online survey creation easy for you. However, beyond the out-of-the-box templates, you can further customize your survey look and feel.

Branding
Making your survey look professional is more than using an existing template. You need to be able to make that template your own with company branding and images that match the theme of your survey questions. A survey that looks unprofessional is a survey that many respondents won't trust. Cvent's customizable tools let you insert your organization's logo and other branding tools into the actual survey.

Added Features
HTML, images and video - three key components that can make a survey feel more interactive and three components that Cvent helps clients use when they conduct survey research. Brighten up that typical text look with colorful images or videos that are relevant to your survey topic.

As we've recognized the importance of providing good customer service in the banking and restaurant industries in earlier discussions, retail is another field where surveying consumers is vital. From clothing stores to music stores to sporting goods outlets, retail surveys are a strong component of making any store better.

If you're searching for a survey provider to help you gather consumer feedback and measure how well your store is performing, it's important to consider how you want to conduct survey research. If you own and operate one local store, telephone surveys may be a good avenue for your research. However, if you're looking to understand how customers that shop in a number of different city-wide, regional or national stores feel, finding a top-notch web based survey provider is the answer.

Find out how your customers feel about in-store selection. Learn which stores are providing world class customer service. Discover how you can improve.

Gathering customer contact information is simple for retail stores. Train your employees to ask customers for their email addresses and telephone numbers, and you can use this information to distribute sales information and request survey feedback. Just be sure to select a web based provider that automates your compliance with the CAN-SPAM act.

With well-crafted retail surveys, you can reveal insights into what you're missing and what you can do better to make that annual report include bigger earnings.

In the crowded online world, concerns over data security continue to grow. When you choose a web-based survey provider, it's essential to find one that keeps respondent contact information and feedback safe.

Password Protection
Sometimes when you send out your survey, you want to make sure that only authorized respondents can take it. By selecting a feedback management solution with a survey password protection option ensures reliable data collection and guarantees that your results will be as accurate as possible. Why would you want to lock down your survey with a password? It's a fair question since most of the time we're trying to figure out how to get more people to complete our surveys. A good example of why you would want to password protect a survey is with employee job performance assessments. You may want to only give managers the password to the survey, so if the link somehow makes it into an employee's hands they still wont be able to access the survey questions and skew the results.

You also should have the ability to lock down the survey based on who you've invited to complete it. This only works with known surveys, however, you can limit the survey respondents to be only those who are on your contact list and received a email survey invitation. This method prevents people from forwarding the survey to others. Need another reason why this is useful? If you're offering a survey incentive, and only have 50 to give away, you don't want your survey respondents forwarding their email invitations to their friends. 

Another valuable component of a survey solution provider is the ability to integrate text verification tools in your survey. Make sure that a live person is answering your online survey, not a spam robot.

Limited Response Capabilities
Worried about multiple responses from the same respondent? Before you begin creating surveys, make sure your survey software lets you limit the number of responses from specific IP addresses or use cookies to ensure results will be even. This will give your organization feedback it can use to improve service.

Data Storage You Can Trust
Conducting survey research means gathering a lot of information. Do yourself a favor, and pick a survey provider that gives you complete confidence with its data storage capabilities. Make sure all of your data is frequently backed up on multiple servers and housed in a SAS-70 environment. After all, that survey research takes a long time. You don't want to lose any valuable feedback.

Encryption for Added Security
To reassure potential respondents that their contact information is safe, be sure to pick a provider that uses SSL technology and HTTPS secure web pages. These added security features encrypt all your information to protect it from third-party viewers. Make it clear within your email invitation or the survey itself that all contact information and answers will be kept confidential and will adhere to your organization's privacy policy.

We've all been part of a depressingly unproductive meeting at some point in our careers – you might even have been part of one this week, and it's only Tuesday. While some meetings are unnecessary and could be replaced with emails, not all meetings can be replaced with thought out email exchanges. One of these are survey planning meetings. Some parts of the survey research process can be limited to email exchanges and short conversations over the phone or face-to-face. However, depending on your survey, there's a definite need to sit down to discuss it – both in the beginning stages and at the end when you are sharing survey findings.

When you do need to have a meeting, respect everyone's time and to-do lists by keeping these four things in mind:

Set meeting objectives before you enter the conference room. If you're leading the meeting, be sure to set clear expectations beforehand – this includes letting everyone know your goals as well. By letting everyone know ahead of time, they can spend some time thinking about the topics. It can help to take an extra few minutes to jot down an agenda to keep yourself focused.

Get everyone involved and participating in the meeting. Chances are if you invited someone to the meeting, there's a reason they're there. This should be, in part, to contribute and share their ideas. Be sure to create an environment where everyone has a chance to share their opinions. Don't shoot anyone down, it's your job to get the ideas and widdle them down later.

Watch the clock and stick to your time line. When you send out a meeting request, you set an expectation for how much time you'll take, be sure to stick to that time frame. If you plan appropriately, you should have no problem getting all of your survey planning objectives accomplished so you can move on to the next piece of the market research process – whether that's designing a survey form or creating survey powerpoint slides for a presentation to the board. You'll find if you stay on topic, most of your meetings should fit into a half hour block.

Define next steps. Make sure that at the end of every survey meeting, you know what the next steps are and who is responsible for each activity. Someone should send an email immediately after the meeting so everyone has a reminder of their tasks. Sometimes you need to have multiple people in the survey designing process from creating survey graphics to writing survey questions to list generations to final approval.

Not every meeting needs to be a waste of your time. If it's about an important business process – like all customer surveys or employee feedback questionnaires should be – then it's worth spending a half hour discussing and ensuring everyone is on the same page. If at the end of the survey research the deliverables are completely different than what was expected, you may have wasted all that time writing survey questions, completing survey data analysis and creating professional survey reports.

If you've created an online survey and used email marketing tools to attract respondents, you probably already know that automated survey tools are some of the most effective tools.

When you conduct internet survey research, chances are you may have a large pool of respondents but a small open rate for your email survey invitations. The growing world of email marketing teaches us that one message is often not enough. Because the typical respondent receives hundreds of other email messages each week, your survey invitation may be looked over on the first glance.

Automated email survey tools can help you make your email marketing efforts work harder. When you search for an internet survey solution, be sure to see what features are included. Can you set a timer for respondents who haven't filled out the survey? Will the timer automatically resend your survey invitation with a new subject message? Can you send a different message to people who have started - but not completed - your web survey form? Can you enable automatic thank you emails for individuals who have already responded?

Simplify your job when you create your next web survey. Use automated survey tools to lighten your workload.

One of the most common pitfalls of data collection is the lack of a clear objective. Before you dive into the world of web based surveys, make sure you define what you are trying to learn about your customers, your product or your employees. Much of this research requires working backward. To conduct effective research, you must recognize what type of data you need for comprehensive analysis.

Let's consider a struggling shoe company as an example. If the company is conducting market research to understand what type of shoes they can create to increase their sales, they must first decide what audience they want to reach. Do they want to sell to males or females? Young adults or an older crowd? Urban or rural areas?

Once they have decided their target demographic, it's important to ask very specific questions. Vague questions do not yield the results you need.


Sample Product Survey Question
Instead of asking a simple yes or no question, an effective survey will help the company learn which attributes will make their target market (in this example an 18 - 25 year old male) spend over $100 on athletic shoes. The example product research survey questions below can help the company make decisions about their shoemaking and their marketing campaign for the specific product.


Sample Product Survey Question
An online survey is one of the most effective market research tools available - when used properly. Be sure to understand what type of data will make an impact on your organization before you create a survey.

No survey respondent wants to devote an hour to a survey. If you keep your online survey concise, chances are your response rates will increase.

Timing Is Everything

If you limit your survey to under 15 questions, it should only take 5 - 10 minutes to complete. Be clear about the length of the time respondents should expect to spend on filling out answers. If they know they can give their feedback in under 10 minutes, they're less likely to abandon your web questionnaire.

Page Breaks

A survey form that looks overwhelming is a survey form that doesn't get complete. When you create a survey, break up your answers into shorter pages that fit on each screen. Rather than open a survey that appears to be a mile long, respondents will see 3 - 4 questions on each page. There is the added benefit, depending on your online survey software tool, that even if someone abandons the survey before completing it, the answers from previous pages will be saved.

Status Update

Putting a progress bar at the top of each page of the online research survey helps respondents track how much more time the survey will require. An accurate percentage measurement helps survey respondents recognize that they are almost finished. There are also disadvantages to using progress bars, so make sure to weigh the pros and cons first.

Sample Surveys and Templates for Extra Guidance

Writing unbiased questions from scratch can be challenging for survey beginners. When you choose an online survey program that offers templates, existing questions and easy-to-use answer forms, you can make sure that your survey research is simple to set up. Using pre-tested surveys helps you avoid creating vague and long questions and they deliver the data you need.

Yesterday, I talked about utilizing incentives for longer than typical studies, or longitudinal surveys.

In addition to splitting our total incentive amount throughout the entire effort rather than all at one time, another key issue to think over is whether you want your incentives to be conditional or unconditional. 

As we discussed in Part I, survey research that looked at incentives for longitudinal surveys was published in the Journal of Official Statistics (JOS). One idea for using incentives in longitudinal efforts is to make your first incentive offer conditional (that is, participants only receive the incentive AFTER they provide data) and the second and subsequent offers unconditional (providing the incentive regardless of whether or not participants provide information for the effort).

Mixing up the way you offer incentives for your longitudinal efforts may decrease the amount of incentives that are "lost." That is, money or products that are given out to those who do not respond at all. A lot depends on the survey response rate you expect at your first wave (and to a lesser extent, your data collection method).

As you can see, there's a lot to think about when providing incentives to participate in your longitudinal survey, but if planned out, your research can benefit greatly from them.

What if all I did was change the background color to improve the readability of my question text? Or I changed the font size and type to make the items more or less legible in my web surveys? These choices may seem irrelevant, but they will affect how fluently your participants process the words in your question items and other survey materials.

Research has found that participants were more willing to add an exercise into their daily routine merely by receiving the directions in an easy-to-read font. When the identical directions were in a difficult-to-read font, participants thought the exercise would take nearly twice as long (8.2 minutes estimate versus 15.1 minutes).

So when we translate these findings to the field of survey research, we can see how clear, easy-to-follow directions and text in our web questionnaires are very necessary if we want our clients to participate in our surveys, instead of simply deleting the invitation emails. Since choices are affected by such small things like colors and font styles or sizes, as survey creators we must always consider how these things may affect our survey response rates and the quality of the data collected. 

Survey participants have to feel they can do a task. Since the presentation of your questions and survey desin may sometimes be more important than the questions themselves, ensuring your survey is readable and in visually stimulating colors will help your respondents feel like your survey will be do-able. This will in turn help them more fluidly process the survey they are being asked to complete, leading to more use able data for your company.

We're always trying to make it easier to create surveys online and find new ways to help you design questionnaires that yeild better data. A few weeks ago we rolled out some pretty impressive new features. One of my favorites is a new report that shows you where survey respondents abandon your online questionnaire. Doesn't seem like it would be a big deal to know at what point people decide to stop completing your survey, but it can really help you design surveys online more effectively.

Here's why you need to know when people leave your survey, it's pretty simple:


People abandon surveys for lots of reasons: it's too long, it's not interesting, it's not relevant, it's too personal. If you know where people jump off, you can narrow down the reasons they stop. With Cvent's enterprise feedback management solution, you can see exactly where they leave the survey form. From there you can look at the next set of questions you ask and see if anything might be either wrong with the survey, too personal or irrelevant. If survey participants abandon your quest for feedback in the middle or toward s the end of a long survey, survey length is probably your first guess.

It's particularly important to think about where your survey respondents during the test portion of your survey research project. Online survery best practice states you should always test your survey, and your messaging, to 10% of your survey sample. If the majority of that test segment is dropping off in the same place, you've got a problem you'll want to resolve before sending email survey invitations to the other 90% of your market research sample.

Next time you're working on designing online questionnaire forms, make sure you review this report and tweak your survey question flow so you get the maximum reliability from your survey data.

If you’re going to use survey assessment tools, remember that just because you want someone to take a survey doesn’t mean they will. Whatever information you’re trying to gather with your survey research software, that information means much more to you than to the person you’re trying to gather feedback from.

Offering it as an online survey is a huge step in the right direction. Enterprise survey software tools make it faster and easier to conduct a feedback survey; faster to build and put out in the field, faster for survey respondents, and faster for survey reporting and data aggregation. Just because you choose to conduct the survey online versus a paper questionnaire, it's unlikely that will get all the responses you want. Here are some additional ways you can make your online survey more appealing to increase survey response rates:

To complete our market research defined discussion of survey sampling errors, we will look at response errors. For those that have read the previous postings on this topic, you have seen how sampling errors are comprised of both non-response errors and response errors. Further, we looked at non-response errors resulting from unintentional exclusions in the market research sample frame or explicit decisions by those contacted to not participate in a survey. Response errors, on the other hand, arise from people taking the survey but the resultant answers are incorrect.

There are generally three types of response errors:


Measurement error results from the survey research instrument itself. Ambiguous and confusing questions can lead to respondents providing information that they believe is true but is in fact not true. They may lack an understanding what the surveyor intended to ask. This can be avoided by making sure that questions are clear and easily interpreted. In interviewing instruments, clear instructions and rigorous standards of interviewing will help alleviate measurement errors. In questionnaires, proper grammar is often the key. One way to mitigate instrument problems is make sure to pretest, whether it’s a questionnaire, focus group script or interview guide.

Recording and analysis errors are a matter of surveyors entering incorrect data into the survey database. Many years ago data processing errors could occur by incorrectly producing those IBM keypunch cards. Now, data entry is often the result of keying incorrect data in to a computer database or incorrect programming of automated data capture systems.  A good way to avoid some data entry errors is by utilizing computer programs that can check logic consistency across answers to survey questions.

Finally, respondent error occurs when respondents provide misleading information. This may happen intentionally or unintentionally. Respondents may not want to admit to certain behaviors or opinions. A misunderstanding of a question may also lead to respondents giving an incorrect answer. Researchers need to be aware that there are both sampling errors and non-sampling errors.

Reporting sampling errors is fairly straight forward and easily quantifiable. However, non-sampling errors, both response and non-response, are also important to understand. Every effort should be made before actual survey data collection and entry in order to minimize their occurrence.