I ran an experiment recently. I gave ChatGPT and the Lean Coach AI I built the exact same opening statement about a real problem.
Then I asked each one for help with creating an A3.
The difference in how they responded tells you almost everything you need to know about why “AI coaching” is harder than it looks — and why the version that feels more productive might actually leave you worse off.
The Same Problem, Two Very Different First Responses
I started both conversations the same way:
“We're struggling to get our plant managers out onto the shop floor for their leader standard work practices.”
The Lean Coach responded with a question:
“What gets in the way most — schedule, priorities, or something else?”
Short. Focused. My turn to think. When I answered — saying the managers would cite schedule and time pressures, but that I also suspected they weren't fully bought into this style of leadership — the coach reflected that back and probed further:
“That sounds more like ambivalence than opposition… When shop floor visits do happen, what do you notice in how they're done — are they just walk-throughs, or is there any coaching or engagement with the teams?”
Two exchanges in, and even though I was just testing the system, I caught myself actually thinking harder about the problem. That second question would catch a real user off guard — it's the kind of question that makes you realize you don't have a crisp answer for something you thought you understood. You hadn't really looked closely at what happens during the walks that do occur.
That slight discomfort — the moment where you realize you don't know the thing you thought you knew — is exactly where learning starts. It's not a comfortable feeling, but it's a useful one.
The coach hadn't told me anything. It was trying to draw out observations and started to challenge whether the obvious explanation (time pressure) was really the whole story.
ChatGPT took a different approach. Its very first move was: “Yes. I'll assume the issue is:” and then it restated my problem for me in bold text. No questions. No curiosity about what I'd observed, what I'd tried, or what I actually knew versus what I was assuming. It just decided it understood the situation and started producing output.
That right there is a red flag. A good coach — a good listener, really — hears a problem statement and gets curious. What have you seen? What's been tried? How do you know? ChatGPT heard a problem statement and started solving.
“Help Me” Doesn't Mean “Do It for Me”
After a couple of exchanges, I asked both for help creating an A3. I want to be specific about the language I used:
“Help me create an A3 on this step by step.”
Help me. Two words that imply a pretty clear division of labor. I'm doing the work. You're supporting me. Step by step means one piece at a time.
The Lean Coach heard that and responded correctly:
“Let's start with Section 1 — the Title. The title states the problem you are addressing. Not a solution, not a project name — a clear statement of the problem or gap. What problem are you working on?”
One section. One question. My turn again. I'm the problem solver. The coach is keeping me on track.
ChatGPT heard “help me step by step” and apparently interpreted it as an invitation to generate every step — all at once, completed for me. Without asking me a single additional question, it produced all ten sections of a full A3.
Here's a summary of the exchange, with a PDF of the full chat… so you can see: How. Much. It. Said. It's exhausting.
Four title options (with a recommendation). A background section. A current condition breakdown with “expected” versus “actual” — both written by ChatGPT based on nothing I'd told it. A problem statement. A target condition with sample metrics. A root cause analysis with five cause categories, some containing four or five sub-points each. A complete 5 Whys chain that ChatGPT answered by itself, top to bottom. Five sets of countermeasures labeled A through E. An implementation table with actions, owners, and due dates. Follow-up measures, including a list of things to watch for as potential bad side effects. And then a full one-page A3 narrative pulling it all together.
So many words. Every section filled in or templated out with blanks for my numbers. Speculative answers to questions it never bothered to ask.
And then — only at the very bottom, after the entire A3 was already written — ChatGPT said: “Send me these six items and I'll turn them into a solid draft.”
Six questions. Asked after it had already produced the draft. By that point, what's the person supposed to do with those questions? The AI already answered them. It guessed at the causes, guessed at the countermeasures, and built an implementation plan around those guesses.
I said “help me.” What I got was a completed assignment. And that substitution — production in place of development — is so seamless that most people wouldn't even notice it happened.
The Comfortable Feeling of Having Finished
Here's what makes this tricky. ChatGPT's response feels productive. You get a big document. It's well-structured. It covers everything. There's a satisfying sense of completion — of having something to show. You could paste it into a template, fill in a few blanks, and present it tomorrow.
The Lean Coach's response feels slower. Less certain. Maybe even a little uncomfortable. You're two questions in and you don't have a document yet. You have more questions than you started with. That can feel like you're not making progress.
But the thing that feels like progress often isn't. And the thing that feels slow often is.
The person who runs with that ChatGPT-generated A3 risks anchoring themselves to someone else's cause analysis — or, more accurately, to a machine's speculation about causes it never investigated. They'll defend those causes in a review meeting. They'll implement countermeasures they didn't think through and don't fully understand. They're not just failing to learn. They're importing false confidence. And false confidence might actually be worse than a blank page, because a blank page at least has the honesty of admitting you don't know yet.
Why This Distinction Matters
John Shook, who literally wrote the book on A3 thinking at Toyota (Managing to Learn), said something I come back to often: “It takes two to A3.” He meant the author and the coach. The coach doesn't write the A3. The coach asks questions. Challenges assumptions. Pushes the author to go look, to gather facts, to think more carefully about what they actually know versus what they're guessing.
As I wrote in Lean Hospitals, “The best A3s are iterative documents, continually refined and adjusted as the author and team better grasp the situation and root causes, frontline workers provide additional feedback, and the mentor asks challenging questions and provides constructive feedback.”
A Lean facilitator at one hospital put it this way: she always knows an A3 author is on track when they have to rewrite something after sharing their progress with people.
That rewriting is where the learning happens. Not in receiving a finished document from an AI that guessed at your root causes.
ChatGPT skipped past thinking entirely and went straight to producing the deliverable. It treated an A3 like a form to be filled out rather than a thinking process to be worked through.
The Real Output of an A3
A lot of people treat A3s as documents — things you produce and present. But the document is a byproduct. The real output is the thinking. The learning. The development of the person doing the problem solving.
John Toussaint, who led the Lean transformation at ThedaCare, said “A3 thinking was critical in my development as a Lean leader.” Not A3 documents. A3 thinking.
When an AI writes the A3 for you, it short-circuits all of that. You get a nice-looking artifact with no learning behind it. It's the equivalent of a consultant who flies in, talks for 45 minutes, hands you a report, and says “any questions?” on the way out the door. Everyone in the room feels like something happened. Nothing did. The organization didn't build any capability. The person didn't develop any new thinking habits. The next time a similar problem comes up, they'll be right back where they started — asking ChatGPT to do it again.
And when the Lean Coach asks you a question, you're the problem solver. You're the author. You're the one developing your thinking. When ChatGPT hands you a completed A3, you're… what, exactly? A reviewer of a machine's homework? That's a different role entirely, and not the one that builds capability.
I Fixed a “Tell Me” Loophole in My Own Model
Writing this post forced me to look at my own tool more honestly.
The Lean Coach has two modes: “Coach Me” and “Tell Me.” Coach Me is the mode I've been describing — it asks questions, one section at a time, and makes you do the thinking. Tell Me mode is for learning about Lean concepts.
You can ask “What is A3 problem solving?” or “What are the sections of an A3?” and get a clear, substantive explanation drawn from Lean Hospitals and other source material.
But during testing, I found a loophole. If someone in Tell Me mode said “Create an A3 for me [related to a particular issue]” instead of “Explain A3 thinking to me,” the coach would comply. It would produce a full A3 document — complete with fabricated data, guessed root causes, invented countermeasures, and made-up percentages. Authoritative-looking speculation dressed in A3 format. Exactly the problem I'd just spent this whole post criticizing ChatGPT for.
So I fixed it. The Lean Coach now draws a clear line: teaching the method is what Tell Me mode is for. Doing the thinking for you is not. If you ask Tell Me mode to write your A3, generate your root causes, or draft your countermeasures, it'll redirect you to Coach Me mode — because the value of these methods comes from working through them yourself.
One of the early test users put the problem well. They said they keep seeing A3s at work that look like somebody told an AI to write them. The thinking is absent. The specifics are vague or generic. The whole document has that slightly-too-polished, slightly-too-complete quality that comes from a machine guessing at your situation. And it's too wordyk as AI is prone to do.
They didn't want another AI tool that does the same thing. They wanted one that would actually help them think.
That feedback stuck with me. It's easy to build an AI that produces impressive-looking output. It's harder to build one that resists the urge to show off what it knows and instead asks you what you know. I'd rather the Lean Coach feel a little slower and a little less flashy if it means the person using it actually learns something.
What I'm Not Saying
I'm not saying ChatGPT is useless. It's good at lots of things — drafting, summarizing, brainstorming, generating options. I use AI tools regularly (mainly Claude these days).
But coaching isn't about producing correct content. It's about developing people. And on that front, ChatGPT didn't just fall short — it did the opposite of what a coach should do. A good coach makes you think harder. ChatGPT made my thinking unnecessary.
The content ChatGPT produced wasn't even wrong, really. The template was reasonable. The cause categories were plausible. That's what makes it dangerous. It looks like an A3 without any of the thinking that makes an A3 worth doing. Someone could hand it in and feel like they'd done the work. They'd be fooling themselves, and probably their boss too.
Try It Yourself
Give it one problem you're actually working on. Not a hypothetical — something real that you're wrestling with right now.
Try ChatGPT first. Ask it to help you work through an A3. Then try the Lean Coach at markgraban.com/start or the Lean Hospitals Coach at leanhospitalsbook.com/start.
You'll feel the difference in the first two questions. One makes you feel like you're done. The other makes you realize you've just started. Pay attention to which feeling is actually more useful.
This lean blog post shows the difference between doing the work versus developing real problem solving skills. This blog talks about how Chatgpt and the Lean Coach have differences in how they complete processes. Furthermore, Chatgpt completed an entire A3, but skipped crucial steps like questioning and collecting data. These steps are essential to complete root cause analysis. Then, the Lean Coach followed a more DMAIC approach by asking questions and gathering data which helped the thinking process. This implements tools like A3 and six sigma which produces a problem solving ability, rather than just answers. Something that I learned was that relying on AI too much can miss crucial parts and create false confidence which leads to inefficiencies.