On NPR recently, the hourly news update covered economic indicators:
Gas prices are UP 5 cents from a week ago Gas prices are DOWN 10 cents from a month ago
So gas prices are both going up and down — depending entirely on which data point you start from. Both statements are mathematically correct. Neither one means anything.
This is what two-data-point thinking looks like. It's not wrong, exactly. It's just useless — and sometimes worse than useless, because it prompts a reaction to noise.
If a hospital's margin is “23% higher” than the year before, is that a difference between 1% and 1.23% or the difference between 10% and 12.3%? Was it down 27% the previous month? Possibly. Some metrics simply fluctuate around a stable average, and 23% can sound dramatic while meaning nothing at all.
Reacting to this “noise” and trying to explain it is a waste of time. That's why the cover of my book, Measures of Success has rollercoaster imagery on the front cover. That's why the subtitle is “React Less, Lead Better, Improve More.”
It probably should have been “Using Process Behavior Charts to React Less, Lead Better, and Improve More.”
Think about what actually happens in organizations when a metric moves.
Someone has to explain it. A manager asks, “Why did this go up?” or “Why did this go down?” and a room full of people reverse-engineers a story to fit a number that was probably just noise.
The next month, the number moves back and nobody learns anything — except that their job includes regularly justifying random variation to people with authority over them.
That's not improvement. That's a performance staged around data that doesn't mean what anyone thinks it means. And it happens on a schedule — monthly, weekly, sometimes daily — because the dashboard refreshes and someone upstream wants an answer.
The Power of Run Charts
Run charts plot data points over time, providing a clearer picture of trends. By visualizing data, we can see patterns, fluctuations, and outliers more effectively than with tables of numbers. For example, if a hospital tracks its monthly margins on a run chart, it can identify whether the margins are consistently improving, declining, or fluctuating within a stable range.
Run charts also help distinguish between routine variations and significant changes (although Process Behavior Charts, with their calculated upper and lower limits are better at this). This distinction is crucial for making informed decisions. Instead of reacting to every change, we can focus on signals that indicate real shifts in the process.
The EPA provides an interactive run chart that you can play with:
Be careful, though, since run charts can be misinterpreted as much as they are helpful, as I illustrated and discussed in this blog post:
Process Behavior Charts: A Step Further
Process Behavior Charts (PBCs), another name for a control chart (the “XmR Chart”), go a step further by providing upper and lower natural process limits and a calculated average. These limits help us understand the expected range of variation in our data. If a data point falls outside these limits, it indicates a significant change that warrants investigation.
See more about how to create these charts (and how to use them).
For instance, if a hospital's monthly margin suddenly drops below the lower limit, it signals a potential issue that needs attention. Conversely, if the data points consistently fall within the limits, the process is considered stable — we still might need or want to improve it, but asking, “What's wrong with last month's number?” is not the way to do so.
We need to ask about systemic drivers for the long-term consistent underperformance and then think about systemic improvements or breakthrough initiatives that can boost the average. That's different than telling people to care more or to try harder in the current system.
Join the Discussion
I posted a shorter version of this on LinkedIn the other day, and it sparked a lot of great discussion:
Related Posts:
See all posts on Process Behavior Charts and a few of my favorites:
