The idea
Define customer segments before assignment, then randomise comparable customers within each segment to offer and control groups. A 10% control is illustrative, not a universal sample-size rule: use baseline outcomes, expected effect and variability to choose the size. After an appropriate observation period, compare contribution per assigned customer, including offer costs and relevant customer outcomes. A 90-day window is an example.
Use it thoughtfully
- Hold out a random 10% control group.
- Match offer value to segment lifetime value.
- Measure contribution after 90 days, not redemptions.
An example
ILLUSTRATIVE EXAMPLE: A retention offer shows 40% redemption, but the control group stayed at almost the same rate, so the offer is redesigned for higher-risk customers.
A question to try
What was the retention rate of the control group in your last retention campaign?
A useful framework is a starting point, not a guarantee. Effects vary with the audience, product and situation. Use truthful evidence, respect customer choice, and test the result.