Product Evaluation Survey

Completing a study is just the first step in the market research process. A good researcher knows there is plenty of work left to be done analyzing survey data and taking action on survey results.

Stay in touch with the client and schedule presentations of the research. If you have a client contact assigned to the project, go over the preliminary survey data to see what key findings they feel are the most important to share. Create presentations around these key findings. You can also create separate market survey reports for different groups within the same company.

For example, if you have just completed a large product survey that includes feedback on customer satisfaction, the client may want a presentation simply on customer feedback for their customer service team.

A good researcher also has an eye for finding problems. If you notice an area for improvement in the course of your market research analysis, propose a solution to that problem for the client. Take this market research survey example, customers complain about being on hold too long when calling customer service. You could propose several ideas from hiring more staff to setting time goals for staff to talk to customers.

Even though the client may not agree with your solution ideas, ignoring problems you identify through the research is a bad idea. The client is paying you to compile and analyze research data, and they will likely appreciate all your efforts even if the information does not fit in with their current business plan.

I was asked to complete a product market research survey today. The product survey asked me a series of questions about possible products related to improving my beer drinking experience. One of the many things I didn't like about this survey was they never asked me if I thought there was a problem.

For example, one of the possible products was for a different method of carrying beer. Unfortunately, I'm perfectly happy with my current method: keeping the beer in the package it comes in. The suggested product would make me move the bottles from their packaging to take them somewhere. I'm not interested in that extra step. Did they ask me why I wouldn't by the product? Nope. They only asked if I would.

Maybe for their survey research goals, that's all they cared about: Would I buy the product, Yes or No? Kelli mentioned in a post earlier today that it's important to gather product feedback on a smaller scale before launching a bigger market research study. The goal of this market research questionnaire easily could have been to identify which product ideas should be considered for further qualitative and quantitative research. Only products where more than 20% of survey respondents indicated they would be interested in purchasing will have more research around it.

On the other hand, I like to make sure my position is clear when completing attitude surveys. This is probably a respondent bias I bring to the table because I spend so much time sharing online survey best practices. I know that often survey questions are written in a way that doesn't always collect the data the survey creator wants. They just don't always write the right questions. So avoid being "mis-quoted" so to speak, I like to be able to answer a series of question about why I feel one way or another.

Believe me, there were a lot of poorly written survey questions in the feedback form, as well as other issues with their data collection method. I've found that where there is one serious problem with a survey template, there are others hidden as well. If you're worried your survey questionnaire may have problems, go ahead and ask our online survey Professional Services Group to take a look. We're always happy to help.

Looking for sample market research survey questions? Here are a few sample product survey questions you can use in your next product research survey:

Market research is incredibly useful at any stage of the game. Though starting out small may seem like an unnecessary step, sometimes it works out better for you and your client. A thorough study is necessary for any new product or service launch. When dealing with a NEW concept, you may want to conduct a smaller initial study to gauge interest level before continuing.

When conducting the smaller study, set criteria for continuation of the study. Criteria for every client and every study will be different, but the goal of setting parameters should always be the same. Decide what percentage of respondents must be interested in order for you to further develop the concept before continuing the study.

For example, if you gathering product feedback for a major cleaning product company about a brand-new product, you might want 60% of product market research survey respondents to express an interest in order to develop the study further. A full product market research study is extremely time-consuming - the entire project will likely include multiple online product surveys, focus groups and other data-gathering methods.

Market research projects like this have a lot of moving parts - your team will have to write survey questions, compile results and create presentations based on their market research analysis of the data. If it turns out that only 20% of respondents are interested in the new product, it may be better to return to the development phase before continuing with the study.

Even though it might be tempting to forge ahead, particularly in lean economic times, your clients will appreciate you more if you save them time, effort and money by starting out small.

Many product market researchers ask survey questions about consumer behavior.  Often, this includes conducting consumer product surveys that asking how much customers are willing to pay for particular goods and services.  Gathering information about purchasing decisions can help you determine price points, but it is important to remember that how you design your survey questions will affect the validity of the results.  Consider the following tips before finalizing your survey questions about customer willingness and pricing options.

Be concrete.  Use as many specifics as possible about the products and services you ask about, especially if you’re interested in brand or item comparisons.  The more general a question is, the more a customer’s mind can drift or think of categories and items you may not have anticipated.  If you need to define a concept or product, do so.

Use a time frame for reference.  When determining how often someone buys or uses a certain product or service, provide a time frame for them to reference (e.g., How many times do you eat out during an average week?). 

For questions that include possible prices as response options, ask survey respondents whether they would pay a single specific amount (yes or no) rather than asking questions with different amounts as responses.  If they are offered a staggered price list, consumers naturally tend to choose lesser amounts than they might actually be willing to pay in the real world.  In your consumer survey ask some respondents if they will pay price x and others if they will pay price y, and then compare the results of the two groups. 

Just because you find correlation in your data when analyzing survey results does not mean there is causation. I find this is a common mistake in lots of survey reports when someone is new to survey research or conducting data analysis. Take this example:


Some people see this connection as a causation. Customers are satisfied because you contact them frequently to make sure everything is going well. The problem is, it's not a causation. Causation are extremely hard to prove because you cannot control every factor. For example, you may split your territory by industry and your solution suits some industries better than others. Or clients who are really satisfied simply use the product more often so the customer success managers reach out to them more frequently, because they are more likely to have questions. While those who use the product less have less to be satisfied about and may feel they are wasting those budget dollars.

To be able to prove causation, you need to be able to rule out all other possible explanations for the connection. As you can imagine, that's almost impossible to do since we do not control outside factors influencing the survey respondent or even the greater survey sample. Instead, when situations like these occur, we're seeing a correlation between two things. In my customer survey question examples, there's a correlation between how satisfied customers and how often they are contacted.

This principle does not only apply to customer survey research, it also applies to analyzing employee feedback forms, product surveys, market research and any other type of data collection and analysis.

I recently made a few online purchases from a major retailer and was told my order would arrive within 7-10 business days. It has been over a month, and I have yet to see my deliveries! Last night, I received a customer satisfaction survey in my inbox asking me to “rate and review” the products I purchased. Since the products have not been delivered, and I have never even seen them in person, how am I supposed to complete their retail survey?

We all understand online surveys can be an effective tool for gauging customer satisfaction. However, when the survey design for the entire process is not done well, it can create problems survey respondents. By sending me the email survey last night, the online retailer only reminded me I had not received my order.

So, how do make sure your organization avoids these miss cues?

First and foremost, it is imperative you create a process for organizing your survey projects. Find a survey method that will be quick and easy to implement (we recommend selecting an enterprise online survey tool).

Make sure employees understands the process. Even departments not directly involved should at least be aware of the process. For example, if the department in charge of customer surveys had coordinated with the order fulfillment department, maybe I would not have gotten any product surveys before I received my order.

If you plan to use an email survey tool to send retail surveys, ask if the survey software company can integrate with your CRM systems. This will make coordinating between your CRM system and the online survey system a lot easier.

How do you get product enhancement requests? Do you look to competitors for new features or have an open product enhancement questionnaire posted on your website? Do you create product survey templates and survey customers or does your marketing team sit around a conference room coming up with product ideas with no outside input?

All of these are methods companies use when it's time for product upgrades. However, sometimes organizations spend a lot more time sitting around thinking about what would be cool and comparing their feature list to their competitors and not enough time looking to customers and prospects for what they want.

Surveying customers is not a requirement when it comes to listening to their needs at product enhancement time. You have employees on the front line talking to customers everyday, answering their questions, trying to help them create workarounds when features don't exist. You have a sales team talking to prospects daily, sometimes losing deals because your offering doesn't have a critical feature. Do you know what these deal breakers are?

A lot of times this type of information already exists within the organization, but it's not being shared between employees or pushed up to decision makers. My guess is this is due to the organization's culture. So how do you get the information that already exists? ASK employees who are talking to prospects and customers. If you just ask once and they don't give you a response because they don't know, they're probably not prepared - after all, you don't usually ask. You'll need to shift the culture. My best suggestion is to get them to pass the information on as they get it. That way, you don't need to worry about them forgetting a great product enhancement. An easy method to collect these new ideas, host something on your intranet - or create an online survey that only contains a few questions (maybe just one open text box). All the information will be collected in one place for whoever decides what the next product enhancement will look like.

Similar data can also be found, in some cases, in customer market research and customer satisfaction surveys. Depending on the goals of the survey (one should always be figuring out what you can do better), customers will tell you what you're doing wrong. And remember that additional comment box you should have in every survey questionnaire? Customers will write in the things they want if you didn't give them another opportunity to voice those through survey questions.

Looking at competitors and their product offering is important, but it doesn't matter if that's not what customers and prospects are looking for. Keep your ears open and listen to what customers are saying they want. They're using your product, they know where it's failing to meet their needs and how it could do a better job working for them. If you don't already have this information, consider creating a product survey to send to customers.

Survey email alerts, also commonly referred to as triggered email alerts, are a must-have feature for online survey software. Why? So you can respond to customer or employee feedback from HR questionnaires to product evaluation to customer service surveys. Yesterday, I wrote a post on tips for listening to feedback. Timeliness in closing the feedback loop should be a priority for organizations looking to improve how they respond and implement changes in reaction to comments.


A common form for feedback collection is web based surveys. So it should be no surprise when I say you need to have email alerts triggered to quickly respond to customer questionnaire or employee survey responses. Not every survey needs triggered alerts and you don't always need them for every respondent, so Cvent's Web Survey tool gives you three options for survey email alerts:

1. Alerts at the question level. If a client responds to a customer satisfaction survey saying they are very dissatisfied with your product or states they are unlikely to renew their contract, it may be appropriate to set a task for their account manager to follow up. Follow up as soon as possible. I find the sooner you can follow up with someone, show them that you're listening and want to resolve any issues, the easier it will be to win them back. The longer a customer has to think about a problem and stew about the pain it's caused, the bigger deal it's going to be later - possibly a deal breaker.

2. Alerts based on survey score. Many people use online surveys as a lead generation tool, for many of those users scoring leads to help the sales team prioritize follow up is important. In cases like this, you may decide you want to receive an email alert when someone scores over a specified number on the survey to ensure the team is following up with the hottest leads. The same idea could be used for educational surveys where you may want to know when a professor receives below a certain score on teacher evaluations completed by students.

3. Alerts for a completed survey response. There will be cases where you want to know when someone completes your survey and how they responded. Be careful with this, when you have a large survey sample, you don't want to be receiving emails every minute to let you know someone completed the survey.

With Cvent, surveyors have the opportunity to send the survey alert to five people and include a custom message in the alert. If you're using survey question level alerts, you can have different alerts be sent to different people as well. Meaning, if you want your customer care team to know when someone gives good feedback on a call they had, but the sales team to know when someone is unlikely to renew, you can set the alerts up that way.

Sign up for a product demonstration to learn more about Cvent Web Survey software features.

When you're creating an online survey and working on the questionnaire design, don't forget to include an introduction to your survey. This applies to all survey types: product surveys, customer feedback forms, employee questionnaires, customer service surveys, quantitative market research studies, etc.

The purpose is just that, to introduce your survey. An introduction should include some basic parts:

1. Who? Quickly tell the respondent who you are. While they will probably figure it out based on your graphical template, it doesn't hurt to tell them a little bit about what you do.

2. What? Give a brief explanation of what topics you'll be asking about. You don't have to give away everything here, a very high level explanation will do. Here's an example:
3. Why? Explain how the respondent will ultimately benefit from the survey and how you plan to use the results. To continue with the example from above:

Like customer service feedback, product research survey questions that target enhancement requests and ideas are critical for organizations - B2B, B2C and B2G alike. So my question is how do you gather product enhancement requests? I have a couple ideas, and I hope you'll share yours as well. Traditional ways may include enhancement request forms your customer care team completes if a customer makes a suggestion during a help session or front-line employees giving feedback to the developers during a meeting. But perhaps conducting a product evaluation survey among product users might be a good place to start. It's difficult to ask people who are not customers to provide feedback on your offering - let alone provide enhancement requests. It's also a pretty safe bet that some of your customers, if not a lot of them, have some great ideas for how you can improve your offering.

If you do go with some type of customer survey, the question maybe be how do you do it and how often. I would argue the survey could be ongoing if you have customer survey software for conducting online surveys. Customer services reps could add a link to the survey in their signature. Any time a customer has an idea or feedback, they know exactly where they can go to give it. If you're a software as a service (SAAS) organization, a link can be placed somewhere behind the log in screen.

There are various ways you can solicit feedback about your offering - the important part is you do it. Your customers are the ones using your product or service. They have the greatest chance of knowing what they need and how you could satisfy a need or pain they have. I tend to think an organization's job is to solve pain - each organization may target a different pain to alleviate, but they all solve some type of pain.

However your organization decides to gather product enhancement ideas is great, but the most important part is that you ask for customers feedback and you act it. How else do you plan to continually improve your offering to stay competitive in the market place?

I would love to hear how you gather product enhancement ideas.

I was looking through a customer survey template the other day and the first question was "Which of these products do you use?" The options were Product A, Product B, Product X and Product Y. Umm, hello? The point of a customer questionnaire is that you're surveying customers, which means you should already know what product survey respondents bought. Now if you're a B2C organization that distributes products through retail stores, this statement might be a little unfair since a consumer can go into any Target and purchase your product. But for now, let's assume you don't have this channel for product distribution.

Why are you making your business survey longer instead of shorter? You already know the answer, put this information in the contact record. Since you have the data, I would even recommend including the question, hiding it and importing the answers to this question for respondents before sending out any email marketing soliciting survey responses. This way, you can utilized online survey tool features like Pipe Logic. Using Pipe Question Logic allows you to personalize the respondent experience and keep them more engaged. Customer survey best practices tell us more engaged respondents are less likely to abandon your survey, aka increase your response rates.

What other common questions could you eliminate during customer research studies because you already have the data somewhere in your organization?

To copy or not to copy? One of the great features of Cvent's web survey software is the clone or copy survey feature. By copying a past survey you can save time, in not only question creation, but also in the actual graphical survey design. If you've never created an online survey - regardless of the survey application - you may not understand what a time saver this can be. Selecting the right color scheme, loading the correct images, making sure everything lines up the way you want in the web survey template can all eat up a chunk of time, and before you know it it's lunch time - or worse, the day is over and all you've done is design the graphical layout of your survey! So having a clone web survey option is - in my book - a must have. We've said before, when it comes to surveys, respondents judge a book by it's cover. And I think they should. In this wonderful world of technology, there's no excuse for having an ugly looking survey, particularly when you could be reinforcing your brand. Furthermore, ugly surveys do not make for a very good survey respondent experience.


However, I want to warn against simply copying a survey and emailing questionnaire invitations out to an email list. Why would you do that? Perhaps, you argue, you want to run the same product survey template in a different state, region, country, etc. or you want to be able to bench mark against last year's customer service feedback or job satisfaction survey. These are fair points but is there other information you realized you needed to improve your product? Was your customer questionnaire perfect? Could you improve your staff opinion survey? Of course you can! Nothing is ever perfect, and when it comes to surveying improving the quality and reliability of your survey data, you should be striving to get the strongest data possible. You're probably planning to use the data you collect to make business decisions, and with that in mind, why would you ever argue for simply copying an existing survey without evaluating the questions you asked last time and if the questionnaire could be improved?

If surveying is part of your job (HR surveys, marketing surveys, customer feedback surveys, etc.), one of the best things you can do before you creating a survey - especially if you've never done a survey project before - is to learn from others. I always take surveys when I'm asked. I'll admit my draw to taking them is because I'm curious about what they're going to ask, but I also want to see what things they're doing right and what things I should avoid.

Looking at other people's surveys are a great way to get ideas for good survey questions. If you're focus is on customer satisfaction or customer service feedback, it should be easy to put your hands on other organizations' business surveys. After all, we're all someone's customer. This is the same with marketing questionnaires or product surveys. Every once in awhile, you should fall into someone's sample. However, if you're trying to get sample survey questions for an employee evaluation feedback form, staff opinion survey or other HR survey, the internet might be your best friend.

While it's easy to get question ideas from questionnaires in the same category as the one you're working on, don't discount what you can learn from surveys in other categories. Best practices cross over categories and someone creating a customer service survey can learn a lot from an education survey.

If you're interested in learning more about survey best practices, sign up for Cvent's free webinar.

People often worry about the integrity of their survey data and how to protect it. While there's no magic solution, here are a few tips to help you identify those respondents who are not thoughtfully answering your survey questionnaire. Chances are you spent a lot of time creating a questionnaire to meet your survey project goals, ensuring the result is quality responses should be at the top of your list. Here are a few things you can do during the survey creation to ensure the integrity of your survey results:

1. Question respondents' qualifications. Some people refer to this as a knowledge trap. The idea is to verify the respondent is the type of professional they claim to be (and you need for your survey). Typically these questions belong towards the top with other qualifying questions. For example, if you're conducting a market research survey targeting educators you may ask a question specific to their field. But you can also use this tactic for product evaluation surveys or customer questionnaires by asking questions related to your product or service that only a client will be able to answer. Here's a sample question for a Cvent user:


2. Test respondents' logic. These types of questions are intended to catch Christmas Tree-ers and straight liners (those who give the same response to every question to speed things along). For this type of trap, you ask the same question multiple ways. Here are a example survey questions utilizing a logic trap:


Depending on how you use logic traps, it may be necessary to space them out. You wouldn't want to have my sample survey questions appear one right after another. It will irritate respondents because you're asking them the same thing twice and wasting their time. Don't be overly obvious, like I was, about it. When you're looking at your results, if someone said it was very likely they would buy the product in the first question but said it was very unlikely they would buy the product in the second, you probably have a problem.

3. Bring respondent attention back. Sometimes when you're completing a survey or questionnaire online you begin to go on auto-pilot only reading the part of the question or just skipping to the responses - particularly if the survey is long. A way to combat this tendency is to add some attention traps to your questionnaire design. An easy way to do this is to throw an unrelated attribute into a ranking scale. It forces the respondent to stop think for a second about what the question is asking and refocus. Here's an example:


Instead of throwing out a respondent because they fail one of these tests, I would suggest simply throwing out that specific answer. In other words, purge data at the question level not the respondent level. If they fail every test and their completion time is way off, perhaps their response is impacting the integrity of your results. After all, the goal of every survey, questionnaire or feedback form should be to answer a question and use the survey report to make decisions.

When any organization comes up with an idea for a new product or service, they test the concept, right? Well, I certainly hope they do before investing in the development and taking the new offering to market. Gathering product feedback when considering international expansion should be a no-brainer, it's the same for a new product or service.

If you're like a lot of organizations, you might start out with qualitative research. Depending on your target market and budget, you might use online communities, focus groups, customer interviews, etc to gauge your new product ideas. But after the qualitative research, don't make the mistake of thinking you're done with your market research. Organizations that don't include quantitative research in their product development process often see product failure. While there are plenty of benefits to conducting qualitative research, it's not always the best method to get a representative look at the thoughts and opinions of your target market.

It's important to always include some quantitative research data collection method, and by all means incorporate insights and lessons learned from focus groups. My first suggestion would be to ensure that no matter your quantitative research method, make sure there's a high priority placed in selecting a representative sample. Second suggestion, use web survey software to conduct a product survey online.

Whether you're deciding to expand internationally or create a new product, let research help you gather the data you need to make your next business move a success.

Over the weekend, Drew McLellan posted on Drew's Marketing Minute stating it's Time to listen up.  It's what we've been saying all along: organizations have to listen.  Drew explains, "marketers have a tendency to 'talk' a lot.  After all, we have a lot to say.  We have features to point out.  Benefits to reference.  Special pricing to announce... But we also need to listen."  The struggle is always, how?  It's easy to have grand plans of wanting to hear from customers to make adjustments to incorporate their thoughts, but it's harder to enact a program and respond to feedback.  Here are Drew's Marketing Minute tips:

• Client satisfaction survey
• Client needs assessments
• Attend industry tradeshows
• Monitor blogs for your organization and competitors
• Read trade publications
• The old-fashioned suggestion box
• Google yourself/company

It's important to keep in mind other types of online questionnaires and web feedback forms you could use to listen as well, like product evaluation surveys.  If you're a Cvent Web Survey user, you can take listening via online surveys a step further by setting up email alerts for questions answered in a specific way.  You can then follow up immediately with contacts who are unhappy or have great ideas you want to discuss.  An immediate, personal response, how's that for listening?

Organizations can use this same model to enact similar programs to include customers in product enhancements conversations with product surveys.  An employer can host an internal website with relevant information about the industry, their products, and HR information and include employee surveys.  With more and more people using the internet and availability of easy to use web survey applications, what's your excuse for not implementing a similar program to have a conversation about important issues?

Did you know there are 6 steps in the market research process?  While this process speaks directly to marketing research professionals, the process applies to HR, customer or education surveys as well:

1. Identify and define the problem.  Before you start any web survey project, you should identify the key issues you hope to be able to solve.  This step should also include clearly defined objectives.
    
2. Develop the approach. In this step, you need to establish a budget, understand influencing factors such as the environment or economy, decide on sampling and survey methods, and formulating hypotheses.
    
3. Research design. Designing a survey or questionnaire is considered the most important step in any survey process.  Question design takes a lot of thought and time.  We like to say, "If you put garbage in, you'll get garbage out."  This means that if the questions are bad, the data will be bad as well.  During the survey research design, keep in mind sampling methods and data analysis factors you intend to use.
    
4. Collect the data. Don't forget to test your survey before to ensure you're fielding the correct data.  Thankfully, with the help of an online survey tool, this step is relatively painless.
    
5. Analyze the Data. The types of analysis you planned to perform on the collected survey data should have been decided in earlier steps, but after collecting the data you have to actually perform the survey analysis.  Analysis can be performed using survey analysis tools like office programs, such as Excel, or more advanced programs such as SPSS - the complexity of the questions will determine this.
    
6. Report, Present, Take Action.  The final step in the market research process is to present your survey research findings and draw conclusions.  While Step 3 is the most important because it defines the outcome of your survey, if you fail to complete this last step and act on the findings in some way, the previous steps don't matter. 

As I mentioned in the beginning, this same process can be applied to any type of project: product evaluations, customer satisfaction questionnaires, public relation surveys, etc.  If you give each step the attention it deserves, each of your online surveys should be a success.

A recent 2009 survey by Mintel revealed the percentage of Americans who "almost always" or "regularly" purchase green products has remained the same over the past year at 36 percent. The percentage of American consumers who almost always or regularly purchase green products tripled last year from 12 percent in 2007 to 36 percent in 2008.

The survey also discovered 54 percent of respondents would purchase more sustainable, green products if their price point wasn’t so much higher than non-green products. The response skyrocketed to 74 percent when respondents were asked if they would purchase more organic food if it were less expensive.

If part of your business plan is to become more environmentally conscious and begin—or continue—selling green products, surveys such as this one can reveal a lot. For example, perhaps your survey also uncovers that one of the barriers you need to overcome in the marketplace is cost. Your marketing department can share that feedback with the supply chain and production departments, who can then take steps to decrease costs without decreasing margins.