A guide to effective measures
Measure what matters
Effective measures mean measuring the right things is crucial to delivering a responsive and flexible service. The quality and quantity of data collected and how it is used will strongly influence efficiency, effectiveness and sustainability of service delivery as well as informing the need for future change.
Do you have effective measures in place?
Common issues we have observed include:
- Organisations are often data rich but business intelligence poor
- Limited understanding of productivity and capacity
- Lack of focus on process measures to understand backlogs, rework or failure demand
- Frequency of data collection is not sufficient enough to manage day-to-day operations
- Performance is focused on arbitrary targets, which drive the wrong behaviours
Targets vs measures
A measure is not the same as a target, and we often discourage the use of targets in favour of merciless measurement. A measure is a means of understanding true performance based on data about what is really happening. A target is an objective to be reached on a single measure, and here’s why we generally dislike them…
- They are often arbitrary, or pointless
- They are often pass/fail
- They often end up being seen as maximum levels of performance
- They often only address one simple aspect of a complete service
- Targets often reward and incentivise undesirable behaviour and can encourage a culture where target hitting becomes the main objective rather than delivering your purpose
An example of targets changing behaviour
Target: Train to arrive at terminus within 10 minutes of timetabled time
Undesirable behaviours this causes:
- Cancel train if running late (it won’t count towards your target)
- Side-line the late train so it doesn’t hold others up (statistically one missed train by three hours is better than four trains late by 11 minutes – unless you are on the train that was running late!)
The purpose of the train: To take passengers from A to B
How to establish good measures
Measures should always link back to your purpose. When designing measures, follow this line of questioning:
- What are we trying to understand?
- What are we measuring?
- Why is it needed?
- Who will use it?
- What will they do with the information?
- How will the data be collected?
- How will the results be displayed?
- What systems/ mechanisms will we use to collate the information?
- How often will the data be collected?
- How often will the data graphs etc be refreshed?
- Who is responsible for collecting the data?
- Who is responsible for analysis and reporting?
- What happens when the owner is unavailable?
A balanced approach to effective measures
Measures need to be focused at the right level. Strategic measures focusing on outcomes and the effectiveness of service tend to be based upon monthly, quarterly and annual performance. Performance information that tells you how you’ve done are called lagging indicators. Whereas staff that deliver services need to measure what is happening today, this week, so the data can inform appropriate action that will in turn influence the lagging indicators. These we call leading indicators. The mix of measures you should use depends on your position in the organisation.
At the top, you need strategic performance measures showing effectiveness of service for the Executive team. At Management level, you need leading daily information feeds into lagging monthly information. Front line staff need to measure daily performance and process data, highlighting areas for improvement.
A balance of leading and lagging measures are needed to support proactive management of performance and it’s essential that any framework of measurement incorporates:
- Input: Volume | Demand | Profile
- Output: Cost | Timeliness
- Process: Productivity / efficiency | WIP | Decision points | Problems | Errors
- Outcome / Impact: Satisfaction | Quality | Hardest to do! These are linked to your purpose
Within the measures you have, only around 60% should be fixed; we call these core measures. The other 40% will be transient. Transient measures are temporary metrics that are necessary to monitor some emerging situation. It might be to measure the impact of an improvement you have made, or to better understand the scale of a problem you are experiencing. Transient measures are typically collated and reviewed for only a few weeks.
Five top tips for effective measures
Remember that good measurement involves acting on what the measure tells you.
- Establish what to measure
- Ensure you have the right tools to gather the data
- Analyse the data to understand the ‘as-is’ and identify improvements
- Take action to deliver improvement
- Establish whether improvement has been achieved
Here are more of our blog posts to help you with effective measures: