Power Law Performance Subscribe

Library Research Customer value, acquisition and retention

Don't Chase the Customers Most Likely to Leave

Who should get your save offer, the customers most likely to leave or the ones the offer will actually change? In two field experiments, Eva Ascarza found that aiming at the second group worked better.

Eva Ascarza, 2018

Extra drop in churn from targeting by response instead of by risk

Ascarza (2018), p. 81: the same campaign, aimed at the customers it was predicted to move rather than the customers most likely to churn. Percentage points of churn.

What she was looking at

How does your retention program decide who gets the save offer? If it’s like most, a model scores every customer’s risk of leaving and the offer goes to the highest scores.

Eva Ascarza, a marketing professor, put that rule to the test in two field experiments:

  • A prepaid wireless provider in the Middle East. 12,137 eligible customers were split at random, and the treated group got a text offering free credit when they recharged.
  • A North American membership organization. Some renewal letters included a thank-you gift.

Then she used machine learning to estimate how much each customer responded to the offer, and compared two ways of picking who gets it, by risk of leaving or by predicted response to the offer.

What she found

“customers identified as having the highest risk of churning are not necessarily the best targets for proactive churn programs.” Ascarza (2018), abstract, p. 80

With the same campaign, targeting by response cut churn by an extra 4.1 percentage points in the first study and 8.7 in the second. And the customers most likely to leave and the customers the offer would actually change only overlapped by about half.

“retention programs are sometimes futile not because firms offer the wrong incentives but because they do not apply the right targeting rules.” Same

The paper won the American Marketing Association’s Paul E. Green Award for 2018.

What it means

Who’s going to leave and who’s going to respond are two different questions. Some of your high-risk customers are leaving no matter what you offer them. Some of your low-risk customers were going to stay anyway. Your retention money only earns a return where it changes the outcome.

Related work makes the point even stronger. In one experiment, reaching out proactively to keep customers actually raised churn. Within three months, 10% of the contacted customers had left, against 6% of the ones nobody contacted.

A two-by-two grid of customers by what they do with a save offer and without it. Sure things stay either way. Sleeping dogs stay if left alone and leave if contacted. Persuadables leave if left alone and stay with the offer, the only group the offer saves. Lost causes leave either way. Persuadables and lost causes are both highest churn risk, so a risk score can’t tell them apart.
Concept. The four names come from uplift modeling and were suggested by the analyst Nicholas Radcliffe (2007). Some practitioners call sleeping dogs “do-not-disturbs.”

Where it shows up in a revenue team

  • Save campaigns aimed at your riskiest accounts spend a lot of money on customers who were leaving whatever you offered.
  • Weight it by value, too. The customer you want to target is the one the effort changes, weighted by what that customer is worth to you.
  • Keep a holdout. Leave a random slice of customers untreated, so you can see who the offer actually moved and not just who stayed.

Get the week's articles and research in one email

The Power Law Performance newsletter puts the research to work on your numbers, where volume KPIs pick the wrong winner and the signals that sort value from volume.

New articles and research, straight to your inbox.