Control Groups : Why keep some people out of the loop?

Control Groups : Why keep some people out of the loop?

It all seems a bit counter-intuitive: why would you want to stop sending communications to a pot of potentially lucrative customers who have actively asked to receive your communications? Yet in reality we do this all the time with marketing control groups. 

What is a control group? 

You probably have your own idea of what we mean by ‘control group’, but just so we’re on the same page, we’d go with this definition which describes a control group as “… a statistically significant group of customers who have agreed to receive your marketing communications, but who are withheld from your marketing activities.” 

So a control group suppresses a pool of people who have actively agreed to receive your marketing communications.  

Okay…so why should you have them? 

Control groups are the only way you can accurately measure the true success of your marketing activity. Without them you are just estimating, or worse, allowing biases to infiltrate your results. 

We tend to agree with Jim Novo, author of Drilling Down: Turning Customer Data into Profits with Spreadsheets (definitely one to add to your reading list) who said: “…if you’re not using control groups, you are most likely chronically underestimating the sales / visits / whatever KPI you generate.” 

How do you go about creating one? 

This isn’t as daunting as it seems, and is actually quite intuitive: 

  1. Establish the purpose – do you need a control group to prove the success of your marketing as a whole or just for an individual campaign? At the risk of stating the obvious, a control group without a purpose won’t prove very enlightening. 
  2. Ensure the group is a mirror of the test – tempting though it may be to form your control group from your inactive customer base, don’t. For a control group to prove its worth, it needs to be a true representation of your test group, with a  composition that genuinely mirrors the active sample. 
  3. Certify your control group is statistically significant – your control group needs to be large enough to give you confidence in its results (ie that they are caused by something other than random chance). That’s not always an easy thing to achieve, so if you’re struggling for numbers, consider using opt-outs as a ‘next best’ alternative. 

Is there anything else? 

Yes! Before you launch into silencing a group of your customers, there are some things to keep in mind: 

  1. Silence means silence – whatever the point of the control is, keep it sacrosanct. Ensure the group doesn’t receive any communications 
  2. Renew and refresh – ensure the control group is regularly mixed up – we would recommend once every quarter 
  3. No top ups – keep your control group pure! Don’t allow others into the group as this can corrupt the results 

To wrap up: 

  1. Ensure the control is a representative sample of your live base 
  2. Refresh the control group periodically. Monitor the ‘health’ of your control cell and review its results as much as you review your test group results 
  3. Make sure your control really is a control (ie protected from ALL communications) 
  4. Numbers too low for an effective control group? Use opt-outs as an alternative 


Leave a Reply