There are literally dozens of ways respondents can misread and even misconstrue survey questions. Some potential errors may be easily identified, while other errors can go unnoticed all the way up until data collection begins. The possibility of a large number of respondents skipping the same questions, or customers providing invalid feedback because of faulty survey questionnaire design and implementation, are just two of the many reasons you should test-run a survey before sending out the real thing. Conducting a pretest is the single best way to identify phrases subject to misinterpretations (and question design flaws in general).
Pretests (also referred to as pilot tests) are used to test the validity and reliability of individual survey questions, the entire questionnaire, and/or response scales. While carrying out a pretest adds a step to your task list, the benefits of sending out a valid instrument far outweigh the costs of a pretest (which for online surveys usually means only extra time, not dollars).
To conduct a pretest, you first need to select a smaller survey sample that is still fairly representative of your target population. You then have this pretest sample complete their surveys, while providing feedback about your questions, any technical concerns and other elements you feel could introduce bias or error into the process and subsequent results. These respondents will no longer be eligible for actual survey participation since they will have been exposed to the questions ahead of time, but the process is well worth it. The final step in the pretest is to analyze your results and decide how you should revise the survey to make it better and easier to complete.
One big caveat: make sure you use people unfamiliar with your research. If you use colleagues or acquaintances who are knowledgeable about any relevant topics or technical issues, they may overlook correctable errors such as leading and loaded questions, faulty transitions and overly technical terminology.
By taking the extra step of conducting a pretest, you can gain valuable information from a small subset of people. This will allow you to make the necessary and appropriate revisions, and in turn produce valid, reliable survey questionnaires for your formal survey research project.
Pretests (also referred to as pilot tests) are used to test the validity and reliability of individual survey questions, the entire questionnaire, and/or response scales. While carrying out a pretest adds a step to your task list, the benefits of sending out a valid instrument far outweigh the costs of a pretest (which for online surveys usually means only extra time, not dollars).
To conduct a pretest, you first need to select a smaller survey sample that is still fairly representative of your target population. You then have this pretest sample complete their surveys, while providing feedback about your questions, any technical concerns and other elements you feel could introduce bias or error into the process and subsequent results. These respondents will no longer be eligible for actual survey participation since they will have been exposed to the questions ahead of time, but the process is well worth it. The final step in the pretest is to analyze your results and decide how you should revise the survey to make it better and easier to complete.
One big caveat: make sure you use people unfamiliar with your research. If you use colleagues or acquaintances who are knowledgeable about any relevant topics or technical issues, they may overlook correctable errors such as leading and loaded questions, faulty transitions and overly technical terminology.
By taking the extra step of conducting a pretest, you can gain valuable information from a small subset of people. This will allow you to make the necessary and appropriate revisions, and in turn produce valid, reliable survey questionnaires for your formal survey research project.
Powered by Compendium Blogware
Comments for Survey Research Best Practices: Pretesting