Skip to content

What we measure is secondary to why we measure

(Originally published on Medium.)

I have been having many conversations about Key Performance Indicators (KPIs) lately, and often when I object to “KPI culture” people think I object to KPIs and measuring anything. I don’t.

What I mean by “KPI culture” is the habit organisations have of setting the Key Performance Indicators first, as goals to attain, and then trying to figure out the who / what / how that will enable them to reach these goals. This usually leads to measurements and reporting that are not useful, or gaming the system to meet the targets, or both.

(This reminds me of an algorithm that was written for a pancake-making robot to avoid dropping them: in order to optimise time away from floor, the machine worked out all it needed to do was toss them really, really, REALLY high up in the air.True story.)

I like KPIs.

I think indicators by their very nature are useful to *indicate* whether something is changing and in which direction. We need this, to know whether what we are doing is having an impact, to learn and to adjust. But a thorough understanding of the system, of the change we are aiming for, and of what that would look like in practice, needs to happen in order to determine what measures will actually be useful.

Just come across a paragraph by Toby Lowe on the Centre for Public Impact blog that puts it nicely and succinctly. I disagree slightly in that I don’t see accountability and learning as mutually exclusive — but I particularly like the last bit: that the people doing the work should choose the measurement.

“We need to think differently about measurement. We are very exercised by the question “What should we measure?” It’s important, but it’s secondary to the question “Why are we measuring?” If we measure to make ourselves accountable to others (to ‘demonstrate our impact’, for example) we become subject to Campbell’s Law ⁠ — the act of measurement corrupts the process it is intended to monitor. Instead, we should measure for learning ⁠ — and use learning as the driver for performance improvement (rather than accountability). This is the appropriate reason to measure outcomes (or anything else). Measuring in a way that helps us to learn is partly about the effective capture of both quantitative and qualitative data and partly about deciding what to capture in any given context ⁠ — ideally the data should be chosen by the people doing the work.”

Depth measurements painted on the side of a canal

Photo by Miguel A Amutio on Unsplash