I've always considered myself pretty good at math, even if I am a bit rust now, but statistics have always been a challenge for me. So when I first started hearing the term "statistical significance" in conjunction with market research results, I would be really impressed.
Most people make this mistake.
Just because a survey finding is considered statistically significant, it does not mean the finding is important or that it should be used for making decisions. When a survey findings are statistically significant it means the surveyor is very sure the statistic is reliable. Survey sample size has a lot to do with whether or not your survey research will be significant. Hopefully this example will help clarify what I mean:
Orange Corp. is conducting a market research study. As part of the study, they want to identify if there is a difference in how well male and female spell. Orange Corp.'s sample size 1,000 people for their survey research. Males scored an average of 98 on the test, and women scored an average of 100. This is a very small difference, but it is considered statistically significant. Just because the findings are statistically significant does not mean the findings are important. The results are only significant because of the large sample size. If the sample size was reduced to only 25 people, the findings would not be significant.
Small differences in results can easily be defined as being statistically significant if the sample size is large enough. Statistical significance only tells you if there is a relationship, not how strong the relationship is or the decision making value.
Most people make this mistake.
Just because a survey finding is considered statistically significant, it does not mean the finding is important or that it should be used for making decisions. When a survey findings are statistically significant it means the surveyor is very sure the statistic is reliable. Survey sample size has a lot to do with whether or not your survey research will be significant. Hopefully this example will help clarify what I mean:
Orange Corp. is conducting a market research study. As part of the study, they want to identify if there is a difference in how well male and female spell. Orange Corp.'s sample size 1,000 people for their survey research. Males scored an average of 98 on the test, and women scored an average of 100. This is a very small difference, but it is considered statistically significant. Just because the findings are statistically significant does not mean the findings are important. The results are only significant because of the large sample size. If the sample size was reduced to only 25 people, the findings would not be significant.
Small differences in results can easily be defined as being statistically significant if the sample size is large enough. Statistical significance only tells you if there is a relationship, not how strong the relationship is or the decision making value.
Powered by Compendium Blogware
Comments for What is Statistical Significance?