upvote
That is correct. It's just that in most cases in the real world, there is a complex casual network, and many of those variables are not even measurable. So you have to pick a proxy for one or more of the variables and make a metric out of this.

The other problem is that in the real world, we want to make decisions, and the easiest way to make decisions is to have a single metric to judge everything by. With multiple metrics, you get into these debates about subjectivity.

You can get around Goodhart's law if you are able to pick multiple proxy variables and demand that the user optimize them all. And you pick these variables in a way that it's really hard to cheat (i.e. deoptimize the actual intended variable while optimizing the proxy variables). Game designers do this all the time for example, because the system is clean and simple enough to do it.

reply
I think the difference is that Goodhart's law describes how the causal chain _changes_ as a result of management behavior, and in particular the incentives they design for the labor they manage. Incentives are a causal variable for outcomes, and what happens is that people find much easier ways to produce the outcomes you thought you wanted.

Like if you manage a call center and set up KPIs around average call time, reps will start hanging up on customers. Employees could always have done that, and the causal link was always there, there was just no reason to.

IMO the problem is executives want (and perhaps need) their directs to report and track one big number month over month. If you give them five metrics they'll never know if you're making progress or just oscillating between a few local minima. And if each of their ten directs has five metrics, you now have 50 numbers and no idea what time it is[1].

[1]: https://en.wikipedia.org/wiki/Segal%27s_law "A man with two watches never knows what time it is"

reply