Fuckup: Goodhart's Law in practice - tie a metric to a bonus

It might be a familiar situation to you: You have an increasing code base 📈, teams are growing and your ability to deploy is deteriorating 🚨.

What do you do? How can you make sure all developers and product owners take care of quality?

In order to react to changes and fix bugs we wanted to be able to roll out fixes as fast as possible (that was in ~2014). One prerequisite for this was a that all build pipelines for the main branch are green 🟢.

So we set the goal, that at the end of the day, on average all (main) build pipelines must be green (implicating deployable) at 95% rate. We tied part of the yearly 📅 bonus 🤑 on that.

At the time it sounded like a good idea 👍, and we had a company goal for everyone. We had dashboards at the coffee machine, everybody could see the progress.

We had no idea

1️⃣ We did not know about Goodhart’s Law: 👩🏻‍💼. It says:

When a measure becomes a target, it ceases to be a good measure.

2️⃣ People like to game win against the system 🏆

What happened

Some developers created build pipelines, which did nothing besides being green 🤷🏻‍. If it was not enough, they had one pipeline which created other green pipelines and deleted them afterward.

I got to know this practice a few weeks before the official end of the bonus period. I was responsible for that goal. I saw that I fucked up 🤦. Good intentions. Wrong outcome. At this point, I got to know Charles Goodhart.

What did I learn

I will ❌ never ❌ tie such a metric to a financial outcome 💰 again. We could still have the goal, just not tied to money. Instead, talk to everyone and understand why their build is broken and try to remove 🚚 what is blocking them.