Whether your focus is B2B market research or consumer insights the odds are high that you will need to compare data from survey to that of another. This might take the form analyzing various survey waves measuring brand awareness, customer satisfaction or customer retention. We might also need to compare data from two separate sources, perhaps an internal survey compared to data collected from another party.

It speaks to our credibility if we can provide an assessment of the statistical significance associated with the difference between the two data sources or surveys. For example is the percentage of respondents who would recommend Lonesome Dave’s Pizza changed significantly over the last two quarters? If our data is one file then we can use our statistical package to make this assessment. However, if we have summary data in a spreadsheet all is not lost. The formula below can be used to create a significance tester for proportions. This will allow us to make definitive statements about the significance of a difference between two percentages.

You will need to know: 1) the percentages for both groups, 2) the total number of respondents for both groups and 3) a desired level of confidence (typically 95%). The formula in math symbology is cumbersome, but can easily be broken down and incorporated into a spreadsheet.

Step

1. Group 1 percentage (e.g. 60% would recommend Lonesome Dave in Q2)
2. Group 2 percentage (e.g. 46% would recommend Lonesome Dave in Q1)
3. Size of Sample 1 (e.g. 894)
4. Size of Sample 2 (e.g. 842)
5. Percentage 1 x Size of Sample 1 = 539.1 (e.g. 894 x .60 = 539.1)
6. Percentage 2 x Size of Sample 2 = 387.3
7. Sample 1 + Sample 2 (e.g. 894 + 842 = 1736)
8. (Step 5 + Step 6) ÷ Step 7  -  (539.1 + 387.3) ÷ 1736 = .534
9. Step 8 x (1 – Step 8) = .534 x (1 – .534) = .249
10. 1 ÷ Step 3 = 1 ÷ 894 = .0011
11. 1 ÷ Step 4 = 1 ÷ 842 = .0023
12. Step 9 x (Step 10 + Step 11) = .249 x .0023 = .000574
13. Take square root of Step 12 = SQRT (.000574) = .02396
14. (Step 1 – Step 2) ÷ Step 13 = (.60 - .46) ÷ .02396 = 5.97
15. Compare Step 14 to the Z-score for the desired level of confidence

Our result of 5.97 is greater than the z-score for 95% confidence (1.96) therefore our difference is significant. There is a significant increase in the percentage of respondents willing to recommend our pizza provider.