Why developing the perfect KPI is impossible

Welcome to Ascern Advisers: Ascernment - Analysis to stimulate you to find the Growth Potential in your business.

You become what you measure.

This has been the mantra of management for decades, if not centuries. Almost all business leaders apply this, in a variety of forms: KPI’s, OKR’s, CSF’s, KRA’s … the list is very, very long.

Coming from a financial background, this has been drilled into me from day 1: we set targets, measure our performance, and modify our approach accordingly.

But what happens if we’re measuring the wrong things? What happens if we do measure the right things, but other stuff goes wrong?

Today I’m going to look at how our current measurements are holding us back, and why it’s impossible to develop the perfect KPI.

Performance measurement: widespread, but ultimately flawed

(Almost) All businesses have performance measures. You know the drill:

1. Set strategic targets
2. Define your preferred flavour of measure (KPI’s, OKR’s, etc)
3. Derive Operational Plans
4. Work
5. Measure
6. Adjust
7. Repeat

This is drilled into us in business school and reinforced by every person we come across in our business lives. We don’t even think about it any more, it’s just automatic.

But back in 1956, a researcher named V. F. Ridgway pointed out the obvious:

What gets measured gets managed – even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so.

Simplicity

If you’re a fan of SMART Goals you’ll know that the key to making easily implementable measures is to make them SIMPLE. If they’re easy to understand, and easy to measure, then people will have no trouble following them.

And if targets are not met, it must be because somebody stuffed up: either the SMART targets were not properly aligned with overall strategic goals, or the workers just simply failed. Or both.

We all know this too. We have to measure the right things or else we’ll fail. Then everybody needs to just get onboard and

Trust in the Process

But how often have you had to redesign KPI’s because they resulted in unintended consequences? Be honest now – I know I have.

Unintended Consequences

Ridgway’s point was that unintended consequences can result from measuring performance. This can happen if we think the goals are clearly defined, and also when they’re not.

Some fantastic examples of this can be found in the evolving world of AI. Some are funny:

Some of them are actually a bit scary:

But there are also plenty of examples where the goals worked exactly as intended, but gave an overall worse result …

Intended Consequences (with other undesirable outcomes)

This is where the “dysfunctional” aspect of Ridgway’s comments come in. We can have well-designed measures that are achieved, and still end up with an “OH SHIT” moment.

Like the UK NHS which set a measure for performance waiting times, which resulted massive waiting time reductions … and an increase in death rates in heart-attack admissions.

Or Wells Fargo staff secretly issuing credit cards without the customer’s consent to meet sales targets – they met sales targets, but it is illegal, so … it was fined $185m as a result.

And who can forget Amazon’s attempt to optimise warehouse workers’ time using the infamous “Time off Task” measure. It worked, but they had to redesign it to allow staff to go to the toilet, wash their hands, drink water, and speak to a manager.

The Lowest Common Denominator: Incentivisation

Pairing Incentives with simple measures is often a receipt for disaster. Why? Because “you are what you measure” is put on steroids by “you get what you pay for”.

Cause and Effect: Not only do we all have a tendency to oversimplify measures, we make them hyper-specific and simple to understand. Then we refine them even more when they result in unintended consequences, and refine them yet again when they work, but we have to deal with other outcomes that are undesirable.

Let’s take a look at everybody’s favourite – Sales Commissions:

Step 1: Pay commissions on revenue

Unintended consequence: Sales target met by cutting the price per unit. Gross Margin hollowed out, making the sales less profitable (meaning less margin to pay sales commissions).

Step 2: Modify commission to be paid on Gross Margin

Intended consequence, undesirable outcome: Gross Margin met by selling more to clients with bad credit history. Less customers pay on time, reducing cash available to pay bonuses.

Step 3: Pay commission on Gross Margin after receipt from customers

Intended consequence, undesirable outcome: commissions based on Gross Margin targets being met, paid based on cashflow. Unhappy sales staff seek employment elsewhere that does not have such a strict commission scheme. Why? Because people are people, and if a simpler, more guaranteed commission structure is available elsewhere, many will choose that.

The Lowest Common Denominator in this case is the commission scheme that optimises Sales effort and return, plus delivers the highest overall return for the business. The “tightest” commission scheme for the business doesn’t always give the best overall return to the business, because salespeople who can’t succeed while jumping through all the hoops will leave … and replacing them costs time, money, and lost sales. That’s why incentivisation can be a curse – optimal returns to the business do not always mean the highest returns possible (sorry, my accounting colleagues).

Other fun reads on this topic

James Pomeroy’s summary of how measures function – epistically, pragmatically, and politically.

When incentives and bonuses are toxic by Nicolas Vandeput.

The boy who finally beat Tetris … by breaking it.

“Buurtzorg”, the dutch home care provider that created a more financially sustainable care model by ditching KPI’s.

KPI’s can create a culture of fear or anxiety, by Shah Mohammed.

SMH calling out bank staff’s incentivisation to rip off their customers, exposed by the Australian Royal Commission into Misconduct in the Banking, Superannuation and Financial Services Industry.

If you like this newsletter, check out our Ascern Advisers Blog.

👋 Hi, I'm the founder of Ascern Advisers. We do Strategic & CFO advisory for businesses with Growth Potential. DM me or email me at david@ascern.com.au if you want to chat.