We Thought We Solved That: The Art and Judgment Calls in the PDSA Cycle

An executive says, after hearing about a major quality defect:

“You said we solved that. Why'd the problem come back???”

That might sound like accountability. But it's not necessarily Lean thinking.

The “Study” step is supposed to help us learn from what happened. Did the countermeasure work as expected? Did the process change? Did we misunderstand the cause? Did the old condition return because the system made it easy to drift back?

Those are learning questions.

“Why did the problem come back?” can be a useful question, but only if it is asked with genuine curiosity. When it is asked with frustration, disbelief, or blame, people hear something very different:

“I thought you fixed this. Explain yourself.”

That blame dynamic might very well shut down the learning that PDSA is meant to create.

The Word “Solved” Is Doing a Lot of Work

People often say we “solved a problem” the way we'd say we solved a crossword. Puzzles stay solved. It's definitive. You don't come back the next morning to find that 14-Down has changed.

Work isn't like that. Life isn't either.

People at Toyota tend to use the word “countermeasure” instead of “solution.” When I first heard that, I assumed it was a matter of translation, or maybe modesty. It's neither. A countermeasure is what you try against a cause of a problem you think you understand. It's provisional by design. It's the best answer you have until you find a better one or the situation shifts.

A countermeasure is a hypothesis that says we might have a solution. Or it's a hypothesis that we'll make things better. We have to TEST the countermeasure to see.

If you installed a solution and the problem returns, blame says somebody failed.

If you tested a countermeasure and the problem returns, learning says the system just taught you something: the cause was incomplete, the countermeasure was weak, the conditions changed, or the old process was easier to fall back into.

Same event.

Different meeting.

Different culture.

Four Possibilities, and None of Them Require a Bad Problem Solver

Before we decide the team failed, it's worth slowing down and asking a better question:

What might have happened?

That question changes the tone of the meeting. It moves us away from blame and toward learning. It reminds us that a returning problem is not automatically evidence of poor problem-solving, weak ownership, or lack of discipline.

It might mean our understanding was incomplete. It might mean the countermeasure was sound but poorly implemented. It might mean the countermeasure worked, but the system made it hard to sustain. Or it might mean the original problem was solved, and a new condition created a similar-looking defect.

Those are very different explanations. Lumping them together under “somebody dropped the ball” is not just unfair. It's bad management. It prevents us from understanding the process.

Here are four possibilities, none of which require assuming there was a bad problem solver.

Start with the first one: we implemented the wrong countermeasure. We thought we understood the cause and we didn't, or we stopped asking why at the level that felt satisfying instead of the level that mattered. The problem coming back is the test result. Our hypothesis wasn't supported. That is genuinely useful information, and it cost us less than being wrong for another three years would have.

Second: we implemented the right countermeasure badly. The idea was sound. The rollout was rushed, or the training was a slide in a Friday staff meeting, or the new checklist lives in a shared drive nobody opens. So we try again, and this time we watch what actually happens instead of assuming.

Third: we implemented the right thing, it worked, and then we stopped doing it. This is the one I find most interesting. If the problem stayed away while we did the thing and returned when we stopped, we just ran the cleanest experiment available to us. In Measures of Success, I described turning off a countermeasure on purpose to see if the problem comes back, when it's safe and ethical to do that. Sometimes an organization runs that experiment by accident. The result still counts.

The question that follows isn't “who stopped?” It's “why did stopping make sense to people?” In my experience the answer rarely involves anyone's commitment to quality. The countermeasure added work. The person who championed it transferred. Nobody built it into how the work actually gets done. Nothing sustains itself.

Fourth: we implemented the right thing, it worked, and then conditions changed. New EHR build, new patient mix, new supplier, different staffing on nights. The old cause is still handled. A new one showed up wearing the old one's clothes.

A Fifth Possibility: It Might Never Have Left

Before any of those, I'd want to ask a plainer question. How do we know the problem went away, and how do we know it came back?

Often, the whole case rests on two numbers. It was 14 before the project. It was 6 after. Now it's 13. Two-point comparisons are the most confidently misread data in management. A run of good weeks happens inside systems that haven't changed at all. So does a run of bad ones.

Plot the measure over time — a Process Behavior Chart, if you want my preference — and you may find that the improvement you celebrated and the backslide you're now investigating are both routine variation in a system that never moved. That's uncomfortable. It's also cheaper to discover now than after the third round of “retraining” as an attempted countermeasure.

The PDSA Cycle Is Full of Judgment Calls

Nobody hands you the rules for any of this.

How long do you wait before you Study? Wait too little and you're reading noise. Wait too long and you've spread something that doesn't work to four more units. How small should the test be? Small enough that being wrong is cheap, large enough that the result tells you something. When do you stop asking why? There's no bell. When do you declare a countermeasure good enough to standardize, knowing that standardizing it means people will stop questioning it?

Root cause problem solving involves as much art as science, with trial and error alongside the discussion and the data. I mean that as reassurance. It also functions as a warning, because art means judgment, and judgment means being wrong sometimes.

An organization that treats being wrong as a performance issue doesn't get better judgment. It gets people who wait to be told what to do, and improvement projects that close on time with green checkmarks and no evidence that anything improved.

Which points at the system, not the problem solvers. Look at what actually gets scheduled. The Do step has a due date, an owner, and a slide. The Study step has none of those. Nobody blocks ninety minutes on the calendar three months out to ask whether the thing worked. We ask if the action item is complete. We rarely ask if the problem is gone, and we almost never ask how we'd know.

Fix that, and the four possibilities stop being a blame-filled autopsy. They become the agenda for learning.

Back to the Executive

He wasn't wrong to be frustrated. The problem is real. It's back. And the organization still has a process capable of producing it.

But his question can send the meeting in one of two directions.

One direction is blame: Who failed? Who stopped following the process? Who said this was fixed?

The other direction is learning: What did we think would happen? What actually happened? What changed? What did the data show? What did we misunderstand?

That choice matters.

Because if people hear, “I thought you fixed this. Explain yourself,” they will learn something, too. They'll learn to be more careful about what they report, more cautious about what they call improvement, and less willing to surface bad news early.

That's not accountability. That's fear with a project tracker.

A returning problem is not automatically a failure of problem-solving. It may be the first honest evidence that the organization has reached the Study step. And that it might be time to adjust.

The question is whether leaders will treat it that way.

When a problem comes back where you work, what happens first? What's the tone of those questions?