Online Survey Methods

As is the case with validity, there are multiple aspects of reliability in survey research.  In general though, while validity refers to accuracy in question design, reliability refers to the consistency of your results (i.e. the probability of producing the same results after repeated measurements). 

One way to easily conceptualize reliability is to consider it as another word for precision.  Let’s use archery as an example: if you shoot ten arrows at the target and they all hit the same area, your shots can be said to be reliable or precise. 

You can test for and estimate the reliability of your survey questions in a variety of ways.  You could use a test-retest design, where you use the same questionnaire more than once to see if the results remain consistent.  You could also send your feedback form to different survey samples (consisting of similar respondents) and then compare your results for any fluctuations.  Also, if you create a set of customer service questions designed to provide you with a composite scale, you would want the individual questions to produce similar results if they are intended to measure a single concept (an overall customer service scale, for example). 

It is also important to understand the relationship between reliability and validity; results can be reliable without being valid.  If we use our archery example, the shots are reliable, but if they aren’t near the bullseye (an undesirable outcome!) they cannot be said to be valid.  Reliability is necessary for validity, but it is not sufficient alone.  You want to accomplish both objectives: shots that land near the bullseye consistently.  By creating reliable, valid questions for attitude surveys, online market research, employee reviews or public opinion polls, you can be confident that your results are not due to chance.