Loyalty Register glossary
Control group
A randomly chosen group of members kept out of a test or change, used as the baseline for measuring its effect on the members who received it.
In marketing, a control group is a randomly chosen set of customers who do not receive a test, offer or program change, so their behavior shows what would have happened without it. In a loyalty program, the members who do receive the change are the test or treatment group. The difference between the two groups over the same period is the effect of the change. Without a control group, you can see what happened, but not what the change caused.
Updated · By Loyalty Register Editorial TeamRelated reading
What is a simple example of a control group?
In a hypothetical test, a program offers double points on coffee to 90% of eligible members, chosen at random, and keeps the other 10% on the normal earn rate. Over the month, treated members spend $52 each and control members $50. The offer added about $2 per member, not $52, and its cost is judged against that $2. A before-and-after comparison cannot do this: seasons, prices and competitor offers move spending anyway, and the control group lives through the same conditions.
How is a control group different from a treatment group or a holdout group?
The treatment group gets the change; the control group does not and is otherwise treated the same. A holdout group is a control group used for marketing, often kept out of all campaigns for a longer period. “Control group” is the broader term: it also covers tests of program rules, such as a new earn rate or reward price, where the control members keep the old rules.
How do you build and size one?
Assign members at random, before the test starts, from the same eligible population as the treatment group. If the team chooses who gets the offer, it will pick the most responsive members and flatter the result. Split your heaviest spenders evenly between the groups, because a few of them landing on one side can outweigh the effect. Decide the measure and the period in advance.
Small expected effects need larger groups, because the random gap between two groups can be bigger than the effect itself. A large program can hold out a small percentage; a small one needs a larger share.
What mistakes should you avoid?
Contamination, where control members are reached by the offer through another channel. Stopping the test as soon as the result looks good. Measuring too short a period, so purchases that were only pulled forward look like new ones. And comparing against members who chose not to take part: they are a different kind of customer, not a control group. For the full method, see how to measure incrementality.