At one point we all learned our ABCs. With the pace of change currently increasing in the marketing research world we are now in need of a refresher. In this case the A and B refer to the concept of A/B testing and the C is the outcome of that test, better known as the Champion. A/B testing has long been a practice in direct and database marketing for choosing the best performer, be it on response rate, conversion percentage or profitability. In a recent article in Wired Magazine it was estimated that nearly 100% of visits to Google involved some form of split testing.

How can this concept be applied to the everyday world of market research practice? Oh so many ways! For starters, the questionnaire development process offers a fertile place for testing. Readers of this blog have long seen the merits of using testing for improving survey response and engagement. One example might incorporate using two types of scaling to measure attitudes, perhaps a Likert scale versus another format. Other options include comparing two forms of a customer satisfaction measurement or the impact of varying scale coding (positive to negative or vice versa).

Testing is also useful on the front-end of the survey process. Again taking a cue from our colleagues in direct marketing we can employ testing to see if one subject line improves the open rate of our survey invitation over that of another. Invitation copy, incentives, and even time and date of invitation release can all be tested via an experimental design.

The impact of these types of A/B tests can be measured using common statistical software. The key to all forms of testing, either A/B or multivariate, is to ensure a randomized sample to prevent bias. This can be done on the fly if your online survey platform allows for routing to multiple versions of the survey. Alternatively, you can randomly divide your mail file into multiple groups and assign separate questionnaires to each group.

The use of A/B and other forms of testing has long been used by other marketing groups. It is time that we embrace the concept in order to drive continuous improvement in our processes.