When a measure becomes a target, it ceases to be a good measure.
Goodhart’s Law
For leaders in any team or organization, defining (measurable) goals is no easy job. While there are frameworks like SMART criteria to ensure the goals are Specific, Measurable, Attainable, Realistic, Time-bound, it’s still challenging to gain the equilibrium in how things should be measured.
As Goodhart’s Law stated above, any measurement we set for specific goals, it tends to become the target for people to chase after, to optimize for, while ignoring the ultimate business goals.
In a soviet factory, many tiny and useless nails were produced when the measurement was set as number of nails manufactured. Few giant nails were produced when the measurement was set as weight of nail output.
I personally observed many similar instances where people tried to “game the system” for short term gains but lost competitive advantage in the long run
As a student at school, success was measured by grades, thus many students focused so much on memorizing, but ignored learning.
As a mid-level engineer earlier in my career, I was surprised to see talks on “how to game the system” in order to get promoted at one of the FLAG companies. When the number of CLs (code change submissions) becomes the target, engineers would figure out ways to create a large number of code churns to boost the metric up, regardless of whether they had meaningful value, or whether they made the codebase harder to maintain in the future.
As an engineering leader, I started wondering and reflecting, when we recognize engineers for putting down fire and fixing critical SEVs (Site Events) Incidents, how do we identify and measure those who helped to prevent fire in the first place?
I don’t have a good answer, maybe “excellent engineering” should be deeply rooted in our hearts.
So how can we mitigate the effects of Goodhart’s Law?
- Start with being aware of Goodhart’s Law phenomenon, acknowledging that it’s part of human nature and can conspire against you.
- Help subordinates to understand and internalize the background motivation and business goal behind a measurement.
- Associate the performance of these top-line business goals with individual’s bonuses, promotions, etc.
- Use a set of measurements to gauge the success of a product or initiative, instead of using just one single number.
Using culture as as tool, I started helping the teams to realize and understand the differences between:
- How to build this project FeatureX, ProductY or SystemZ.
- What are the measurable key results of such project
- Why should we build this, which top-line business goal does it serve.
The teams were able to learn from top to bottom as listed above, but in planning, we go from the bottom up in reverse order. This serves as the cornerstone to build autonomous teams that move fast, make decisions effectively and self-direct to the ultimate goal.