Scroll down for video, how to subscribe, video, transcript, and more
My guest for Episode #413 of the Lean Blog Interviews podcast is Samantha Riley, the Deputy Director of Intensive Support for NHS England and Improvement. Sam is the author of an amazing publication called “Making Data Count,” which you can read and experience freely online.
Sam and I are “Twitter buddies,” as she said and I follow and enjoy her tweets, especially those using the hashtag #PlotTheDots. We are both users and teachers of (and advocates for) the use of Statistical Process Control charts (aka XmR Charts or Process Behavior Charts) as taught by the statistician Don Wheeler.
Topics, questions, and links related to today's episode include:
- What's Sam's professional background and why it's OK that she's not a statistician
- Why are RAG (Red Amber Green) charts or tables insufficient?
- How do you have the conversation with boards, executives, and managers about RAG being insufficient?
- How to bring people to the table for this learning?
- Why are up/down comparisons and color coding leading to “knee jerk reactions”?
- What is “spuddling“? Why is that a problem?
Spuddling: To make a lot of fuss about trivial things, as if it were important.
- 50% of boards have changed their approach — how has that come to be?
- What language do you use about improving a predictable system?
- How did you get exposed to Statistical Process Control?
- What led to “Making Data Count“?
- What was “the ham sandwich incident”?
- How can we use these charts to look at our weight and health?
- Tweets about this:
- How to articulate the benefits of SPC?
- Her article in BMJ Leader: National Health Service (NHS) trust boards adopt statistical process control reporting: the impact of the Making Data Count Training Programme
The podcast is sponsored by Stiles Associates, now in their 30th year of business. They are the go-to Lean recruiting firm serving the manufacturing, private equity, and healthcare industries. Learn more.
This podcast is part of the #LeanCommunicators network.
You can listen to the audio or watch the video, below. I hope you enjoy it like I did.
Video of the Episode:
Thanks for listening or watching!
This podcast is part of the Lean Communicators network… check it out!
Automated Transcript (Not Guaranteed to be Defect Free)
Introduction and Background
Mark Graban: Welcome to the podcast. My guest is Samantha Riley, joining us from England. We're going to learn about her background and discuss one of my favorite topics: process behavior charts. Sam is the Deputy Director of Intensive Support for NHS England and Improvement. Sam, thank you so much for being here today.
Samantha Riley: I'm good. Thank you.
Mark Graban: I've wanted to do this for a couple of years, and I'm really excited that our schedules finally lined up.
Samantha Riley: We are Twitter buddies, aren't we? That's how we connected.
Mark Graban: I've been a big fan of what you and your colleagues share on Twitter. I try to promote and share a hashtag that I think we're aligned on: #plotthedots. First off, I'm curious about your title, Deputy Director of Intensive Support. Could you tell us what that role entails?
Samantha Riley: I have a small team focused on supporting those organizations in England that have the most challenges. These might be quality challenges, financial challenges, or both. We provide specific support focusing on how they can use their data to make better decisions. We provide a very intensive package of support to those organizations, but we also support any organization in the NHS in England that wants to work with us.
Mark Graban: So intensive support is not to be confused with intensive care?
Samantha Riley: No. This is intensive support to help organizations improve in a whole range of areas. However, it won't be a surprise to realize that data is a common theme; where we see challenged organizations, they often aren't using their data in the best way.
The Role of Data in Organizational Health
Mark Graban: Are there some organizations that tend to reach out because they are in need of “intensive care” at an organizational level?
Samantha Riley: Some organizations would be badged by regulatory organizations as requiring support because they are probably failing a range of targets or aren't doing well financially. However, we also get approached by people who realize that, while they might not be in a challenged position, they could be doing better in terms of how they use their data.
Mark Graban: When we talk about metrics, statistics, and statistical process control (SPC), are you a statistician?
Samantha Riley: I am not a statistician, nor am I an analyst. I've got nearly 25 years of experience in the health service in various management roles. I've always had an interest in data because you can't improve anything unless you have good data. I was introduced to statistical process control about 20 years ago when we first looked at applying SPC to health service data. It had a good impact then, but people stopped doing it. I've been on a personal mission for the last 20 years to make sure we make good use of this approach because it has such massive benefits.
Mark Graban: We are influenced on both sides of the pond by the statistician Don Wheeler. He takes what could seem like arcane methods and focuses on the practical use of data connected to improvement work.
Samantha Riley: Absolutely. I have to translate things into plain English for myself so I can tell other people. We've been very inspired by Donald Wheeler because of that practical application. The resources we developed a few years ago, called Making Data Count, were very much inspired by his approach. We've taken the approach of teaching SPC in a more simplistic form, and that has really paid dividends for us.
The Problem: RAG Ratings, Poor Visuals, and “Spuddling”
Mark Graban: Can you introduce the current state of how leaders look at metrics? We frequently see red and green color-coded numbers.
Samantha Riley: In the NHS, people have been wedded to visualizations like RAG (Red-Amber-Green) rated charts. Green means you're passing the target, red means failing, and amber is somewhere in between. Pretty much all performance reporting you see on hospital websites includes these RAG tables. We also see lots of arrows indicating if a number went up or down this month, similar to the game show Play Your Cards Right. Fundamentally, that is how we've been making decisions at the board level.
Mark Graban: You mentioned “horribly presented data.” What does that look like?
Samantha Riley: We see spaghetti charts–a flood of different colored lines for different years or wards overlaying each other so you can't see what's going on. We also see stacked bar charts, which we call “rainbows of confusion”.
Mark Graban: And there are bad decisions made from metrics shifting from green to red. We see knee-jerk reactions.
Samantha Riley: Exactly. Someone goes into the red, and there is a panic. Resources are sent to investigate what happened, when it could just be a random change in the data. We have a special term for overreacting to trivial things: Spuddling.
Mark Graban: I've never heard that word. What does it mean?
Samantha Riley: “Spuddling” is an old English word defined as making a lot of fuss about trivial things as if they were important. This concept has really chimed with the boards we work with. We even have yellow cards people can raise in meetings to ask, “Are you spuddling?”. It's a fun, subtle way to challenge colleagues, but it drives real behavior change.
Training Boards and Changing Leadership Behavior
Mark Graban: How do you bring boards and leaders to the table for this training?
Samantha Riley: Initially, we asked for guinea pigs–boards that already thought they could do better. We learned a lot from those early sessions about what didn't work. Now, we don't have to market the sessions; it's all word of mouth. Out of 217 trusts in England, we have run nearly 130 board sessions.
Mark Graban: Have these sessions led to changes in their approach?
Samantha Riley: Yes. We haven't had a board tell us they aren't going to change. It's a difficult message to tell a board that the way they've been using data has led to poor decision-making. We tailor the content to their organization by taking their public performance reports and re-running the data as SPC charts. We show them improvements they missed, deterioration they didn't know about, and instances of “flip-flopping” between red and green.
Mark Graban: You recently published an article on this titled National Health Service Trust Boards Adopt Statistical Process Control Reporting.
Samantha Riley: Yes, that paper describes what we've learned and includes evidence from independent evaluations. We know there has been a significant increase in the number of SPC charts used. I'd say about 25 to 30 organizations now have exemplary reporting.
Visualizing Data: Icons and SPC Mechanics
Mark Graban: How do you handle the volume of metrics? A typical board might look at hundreds of measures.
Samantha Riley: Board members told us, “We love SPC, but we can't have a 200-page report with a chart for every indicator”. To solve this, we created icons to summarize two things:
- Variation: Is it getting better, worse, or is it common cause variation?
- Assurance: Will we consistently pass or fail the target?
We also moved away from RAG colors. We use orange for concern and blue for improvement. These colors are colorblind-friendly. Our tools automatically highlight special cause variation using these colors so people don't have to count dots or remember technical rules.
Mark Graban: This allows for a filtering function. If only five out of 100 charts have a signal, those are the ones you discuss.
Samantha Riley: Absolutely. People are replacing RAG summary tables with icon summary tables. They include full charts where there is highlighted special cause variation. We also want to shine a light on the “blue dots” of improvement to celebrate work that has paid off.
Personal Applications and the “Ham Sandwich Incident”
Mark Graban: Once people learn these methods, they can apply them to anything. You mentioned someone plotting the books they read. I've been plotting my weight.
Samantha Riley: Once people understand SPC, there is no going back. Plotting weight is interesting. My partner lost six stone, and plotting the data helped. We found that setting an unachievable target is demotivating, but setting interim, achievable targets encourages you to continue.
Mark Graban: Can you talk about the “ham sandwich incident”?
Samantha Riley: That became a great piece of teaching material. I was plotting my daily weight and had lovely blue dots of improvement. Then, suddenly, there was a massive astronomical data point. The only thing different was that I had attended a cricket match and eaten a ham baguette.
Mark Graban: You learned from that, and that's the key. It could have been the bread or the saltiness.
Samantha Riley: Exactly. I also fell into the trap of stopping weighing myself when I didn't like the direction, only to find it had all “gone orange” when I finally keyed in the data. Ignoring the metric is unlikely to be helpful.
Origin Stories and Future Goals
Mark Graban: Going back to 20 years ago, what was the spark for your interest in SPC?
Samantha Riley: I was a service improvement manager and met Mike Davidge, who was the Director of Analysis for our National Improvement Body. He introduced me to the charts. He is semi-retired now and pleased that I've taken the baton.
Mark Graban: It feels like you've reached a tipping point.
Samantha Riley: Since lockdown, we moved to virtual training, which allowed us to train about 3,800 people since September. My goal is for the whole NHS to adopt this. A recent review showed about 70 trusts still rely on RAG reporting, so we will be targeting those.
Mark Graban: That is exciting progress. I'll put links to the Making Data Count website and resources in the show notes. Everyone should check out #plotthedots. Thank you again for doing this, Sam.
Samantha Riley: Many thanks.
Please scroll down (or click) to post a comment. Connect with me on LinkedIn.
If you’re working to build a culture where people feel safe to speak up, solve problems, and improve every day, I’d be glad to help. Let’s talk about how to strengthen Psychological Safety and Continuous Improvement in your organization.
