A guide to effective measures

Effective measures mean measuring the right things is crucial to delivering a responsive and flexible service. The quality and quantity of data collected and how it is used will strongly influence efficiency, effectiveness and sustainability of service delivery as well as informing the need for future change.

Do you have effective measures in place?

Common issues we have observed include:

Organisations are often data rich but business intelligence poor
Limited understanding of productivity and capacity
Lack of focus on process measures to understand backlogs, rework or failure demand
Frequency of data collection is not sufficient enough to manage day-to-day operations
Performance is focused on arbitrary targets, which drive the wrong behaviours

Targets vs measures

A measure is not the same as a target, and we often discourage the use of targets in favour of merciless measurement. A measure is a means of understanding true performance based on data about what is really happening. A target is an objective to be reached on a single measure, and here’s why we generally dislike them…

They are often arbitrary, or pointless
They are often pass/fail
They often end up being seen as maximum levels of performance
They often only address one simple aspect of a complete service
Targets often reward and incentivise undesirable behaviour and can encourage a culture where target hitting becomes the main objective rather than delivering your purpose

An example of targets changing behaviour

Target: Train to arrive at terminus within 10 minutes of timetabled time

Undesirable behaviours this causes:

Cancel train if running late (it won’t count towards your target)
Side-line the late train so it doesn’t hold others up (statistically one missed train by three hours is better than four trains late by 11 minutes – unless you are on the train that was running late!)

The purpose of the train: To take passengers from A to B

How to establish good measures

Measures should always link back to your purpose. When designing measures, follow this line of questioning:

What?

What are we trying to understand?
What are we measuring?

Why? (purpose)

Why is it needed?
Who will use it?
What will they do with the information?

How?

How will the data be collected?
How will the results be displayed?
What systems/ mechanisms will we use to collate the information?

Frequency?

How often will the data be collected?
How often will the data graphs etc be refreshed?

Owner

Who is responsible for collecting the data?
Who is responsible for analysis and reporting?
What happens when the owner is unavailable?

A balanced approach to effective measures

Measures need to be focused at the right level. Strategic measures focusing on outcomes and the effectiveness of service tend to be based upon monthly, quarterly and annual performance. Performance information that tells you how you’ve done are called lagging indicators. Whereas staff that deliver services need to measure what is happening today, this week, so the data can inform appropriate action that will in turn influence the lagging indicators. These we call leading indicators. The mix of measures you should use depends on your position in the organisation.

At the top, you need strategic performance measures showing effectiveness of service for the Executive team. At Management level, you need leading daily information feeds into lagging monthly information. Front line staff need to measure daily performance and process data, highlighting areas for improvement.

A balance of leading and lagging measures are needed to support proactive management of performance and it’s essential that any framework of measurement incorporates:

Input: Volume | Demand | Profile
Output: Cost | Timeliness
Process: Productivity / efficiency | WIP | Decision points | Problems | Errors
Outcome / Impact: Satisfaction | Quality | Hardest to do! These are linked to your purpose

Within the measures you have, only around 60% should be fixed; we call these core measures. The other 40% will be transient. Transient measures are temporary metrics that are necessary to monitor some emerging situation. It might be to measure the impact of an improvement you have made, or to better understand the scale of a problem you are experiencing. Transient measures are typically collated and reviewed for only a few weeks.