Library Research Metrics and incentives
The Most Quoted Line in Management Isn't Goodhart's
Charles Goodhart was writing about the money supply in 1975. The sentence everyone quotes came from Marilyn Strathern in 1997, and she was writing about British universities. Both describe what happens when a number becomes a target.
C. A. E. Goodhart, 1975; Marilyn Strathern, 1997
What they were looking at
In the early 1970s the Bank of England changed how it managed the money supply, and it came to rely on statistical relationships between money and the wider economy. How did that go? Charles Goodhart, then a monetary adviser at the Bank, described it in a paper for a Reserve Bank of Australia conference in July 1975, and the paper opens like this:
“The subsequent experience of trying to operate this revised system has, however, been troublesome and at times unhappy.” Goodhart, “Problems of Monetary Management: The U.K. Experience,” 1984 reprint
Twenty-two years later the anthropologist Marilyn Strathern was writing about something completely different, the boom in auditing and ratings in British universities. Her paper was adapted from a 1997 lecture at Girton College, Cambridge.
What they found
This is the passage from Goodhart’s paper that gets quoted:
“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Goodhart (1975), as reprinted in A. S. Courakis (ed.), Inflation, Depression, and Economic Policy in the West (1981), p. 116
The relationship the Bank relied on held up as long as nobody leaned on it. Once policy started steering by it, behavior adjusted and the relationship broke.
Strathern’s sentence is the one you’ve heard:
“When a measure becomes a target, it ceases to be a good measure.” Strathern (1997), p. 308
She credits the “Goodhart’s law” label to Keith Hoskin (“Hoskin describes this as ‘Goodhart’s law’”), and she gives her own example from university degrees, where an upper-second-class result (a “2.1”) had become the grade everyone expected:
“The more a 2.1 examination performance becomes an expectation, the poorer it becomes as a discriminator of individual performances.” p. 308
Her broader point was that audit “does more than monitor—it has a life of its own that jeopardizes the life it audits.”
What it means
Both versions say the same thing from different ends. Why is a number useful in the first place? Because it moves with something you care about. Put pressure on the number and people (and algorithms) find ways to move it that don’t move the thing you care about.
So should you stop measuring? No. But the measure you reward is the one that gets gamed first, so it had better be as close to value as you can get, and you should keep watching the measures nobody is rewarded on.
Where it shows up in a revenue team
- Cost per lead was a good measure of marketing efficiency until it became the target, and then the cheapest leads won.
- Approval rate, churn rate and meetings booked are each a decent measure right up until a team is paid on it.
- Algorithms are the purest case. Tell an ad platform what counts as a conversion and it’ll hit that number with no regard for anything else.
- Anchor the target to value, and keep reporting what each cohort returns over time, since that’s the hardest number to game.