Better Metrics & “Understanding Variation” – An Important Topic for Healthcare

Hear Mark read this post (subscribe to the podcast):


KaiNexus taller version of logoA quick announcement before today’s post: On Tuesday, January 24, I presented a free KaiNexus Webinar on a topic that I think is very important… not just for healthcare, for any type of business:

How to Manage Your Improvement Metrics More Efficiently and Effectively.”

Post continues after ad...

You can view the recoding of this free educational webinar today. Today’s post is about this topic…


screen-shot-2016-11-22-at-7-07-33-amMy favorite book, as I’ve written about before, is not a “Lean book” — it’s Understanding Variation: The Key to Managing Chaos by Donald J. Wheeler, PhD

It might look like a book about statistics (and it has an esoteric sounding title), but I’d call it a management book, and a very practical and useful one at that.

See my post from 2013:

The Book that Changed Me: “Understanding Variation”

I’m excited that Don Wheeler is going to be one of the keynote speakers at this year’s Society for Health Systems Healthcare Systems Process Improvement Conference this March in Orlando. I hope to see you there! I’ll be talking about the conference a bit in tomorrow’s podcast with Isaac Mitchell.

You can read an interview with him here. In part…

“What is the most pervasive issue or challenge in health systems management today?

Among the many challenges for healthcare management is the challenge of using data more effectively. We are told to be more data-driven, yet no one provides any effective mechanism for doing so. Big data techniques require extreme sophistication on the part of the user to avoid being misled by noise.”

Why do I think this is such an important topic?

I see a lot of organizations practicing some form of what they’d describe as “Lean Daily Management” or something like that.

There are a lot of good practices that I see, including:

  • Continuous improvement or Kaizen boards (although sometimes, these get overcomplicated)
  • Daily team huddles (although these sometimes don’t have much energy or team participation)
  • Rounding or “gemba walks” (although these are sometimes more ceremonial than helpful) and
  • Daily performance metrics and charts

It’s generally good to see daily performance measures. It’s certainly better than monthly metrics that lag two months behind.

Real-time daily metrics are much more helpful for understanding our system, driving improvement, and reacting to changes in the metrics (good or bad).

But, the way these metrics are done sometimes causes problems… or it’s just not an ideal way to do things.

In my experience, the approach of Lean Daily Management (and performance tracking_ works better when we:

Use “line charts” for time series data instead of what Excel calls “column charts” (often called “bar graphs”). This is relatively minor point in that changes and trends in bar charts are visually harder to interpret than lines.

Post continues after ad...

Avoid using statistically dubious “linear trend lines” that can be misleading (often unintentionally so, but you’re never quite sure). Wheeler would say that a chart should never lead to somebody drawing the wrong conclusion about whether something is improving (or not) or stable.

Avoid arbitrary goals or targets for improvement, which can be demoralizing if management is asking the wrong questions, such as “what went wrong yesterday?” instead of asking “how can we improve our system and its performance, in general?”

The methods that Wheeler writes about (and that I’ve used and taught) aren’t complicated.

But, these methods help managers and teams make better management decisions. That leads to more improvement, better performance, and less frustration.

Managers and teams need to be able to answer important questions, such as:

  • Do we have a stable system with stable performance?
  • Can we predict tomorrow’s performance?
  • How do we know if there’s a data point that’s worth investigating (one that’s good or bad in statistically significant way)?
  • How can we distinguish “signal” from “noise” in our charts and metrics?
  • How do we know if a process improvement effort has led to meaningfully and sustainably better performance?
  • If our current performance isn’t meeting our goals, how can we improve the system? And how do we know if we’ve made a difference?

I’ve written other blog posts that draw on the lessons from Understanding Variation, including:

Consulting Case Studies Need Statistical Validity

Data Without Context Isn’t Very Helpful; Don’t Overreact to Each Up & Down

Managing With Red/Green Charts is Like Driving By Looking at the Rear-View Mirror

What’s Demoralizing? The Colors on a Chart or Not Improving the System?

Upcoming 2017 Workshops

Here in 2017, I’m going to be teaching some workshops about these themes and practical lessons from the Deming “Red Bead Experiment.” See my previous webinar about that with Mike Stoecklein:

Free Webinar Recording: The Real Lessons of Dr. Deming’s Red Bead Factory

Info about those workshops:

One time I’m running this, we’re using the title: “Understanding Variation and Performance Management: Practical Applications of Dr. Deming’s Red Bead Experiment”

And we’re also experimenting with what we think might be a better title (same workshop, continuously improved) under this title:

Better Metrics: Understanding Variation and Managing Performance Using Deming Methods”

I hope you can join me at one of those events. Or, if you’d like to talk about bringing these methods (a hands-on workshop and some coaching) to your organization, let me know.

And, again, check out my free webinar: “How to Manage Your Improvement Metrics More Efficiently and Effectively.”

Please post a comment and join the discussion. Subscribe to get notified about posts daily or weekly.

Mark Graban's passion is creating a better, safer, more cost effective healthcare system for patients and better workplaces for all. Mark is a consultant, author, and speaker in the "Lean healthcare" methodology. He is author of the Shingo Award-winning books Lean Hospitals and Healthcare Kaizen, as well as The Executive Guide to Healthcare Kaizen. His most recent project is an book titled Practicing Lean that benefits the Louise H. Batz Patient Safety Foundation, where Mark is a board member. Mark is also the VP of Improvement & Innovation Services for the technology company KaiNexus.

You might also like More from author


Leave A Reply

Your email address will not be published.