Come one, come all may be a useful approach for some, but in today’s world of online market research it is wasteful. How do we assure we get data from the key groups we want to reach? That’s a simple answer we use quotas.
Quotas are method of limiting who responds to the survey (or portions of it) based upon their responses to screening questions. Typically screening questions include demographics (e.g. age, gender, income, etc.) or firmagraphics for B2B marketing research (e.g. industry, company size, headquarters vs. field office, etc.) along with some measure of brand awareness or usage. In a recent IT study, I used several measures of job role and function as part of my screening criteria.
Quotas are developed before the survey fields by the researcher, in consultation with internal or external clients. For example if the client wants and equal percentage of men and women in the study the quota for gender would be set at 50% male and 50% female. Complex or nested quotas involve more than one variable. For example:
Quota 1 – Gender: Men 50% and women 50%
Quota 2 – Age: Under 25 and over 25
This would yield a study with four cells (two for gender and two for age). One-fourth of the total completes would fall into each cell. Complex quotas can go beyond two variables with two categories to include multiple variables with several categories each. Bear in mind, however, the more complicated the quota scheme the more difficult it may be to reach the total number of completes. Complex quota schemes can also significantly raise the cost of acquired sample.
Why do we use quotas? The prominent reason is to balance reporting. Cell sizes that are significantly different in size hamper our ability to conduct meaningful survey data analysis. When our group sizes are of equal size then our ability to uncover statistical significance increases.
When using quotas there is a potential downside. Namely, when a participant gears up mentally to take a survey, only to find that they do not meet the quota can lead to disenchantment. Use quotas only when necessary, and if possible, provide those that are screened out another opportunity to participate in the survey process.
