Data vs. Facts: Why Lean Leaders Don’t Trust the Dashboard
TL;DR: Data is what gets recorded. Facts are what's actually happening. Confusing the two is how leaders get fooled by faked reports, broken sensors, and decorated dashboards. Deming and Ohno both warned about this in different ways. The fix is to go to the gemba and see for yourself.
Two expressions get used often in the Lean world – one from Dr. W. Edwards Deming and one from Toyota's Taiichi Ohno. They're often quoted side by side, but they're making different points.
What Deming and Ohno Actually Said
Dr. Deming is widely quoted as saying:
“In God we trust, all others bring data.”
He may not have originated the line, but he popularized it. His point was straightforward: don't make decisions based on opinion or hierarchy. Bring evidence.
Ohno said something that sounds similar but pushes further:
“Data is of course important in manufacturing, but I place the greatest emphasis on facts.”
Ohno wasn't dismissing data. He was warning that data and facts aren't the same thing. Data is what gets recorded – entered into a system, printed on a report, displayed on a dashboard. Facts are what's actually happening in the work. Reports lie. Sensors break. People fudge the numbers when they're afraid of the consequences. The actual work, observed firsthand, is harder to fake.
That's why Ohno emphasized going to the gemba – a Japanese term meaning “the actual place” or “the workplace.”
Why the Distinction Matters
Picture a digital sign showing an outdoor temperature of -121 degrees in the middle of winter. Clearly wrong. A person standing next to that sign would feel that it's cold, but not impossibly cold. The data on the display and the fact on the ground don't match. The sign has clearly malfunctioned, producing questionable data.
Now ask: in your organization – factory, hospital, office – what's the risk that someone is making a decision based on a management dashboard that's the equivalent of “-121 degrees outside”? Maybe the bad data is less obviously wrong than that. Maybe it's a quality metric that looks great because defects are being reclassified. Maybe it's a productivity report that hits target every week because the system is being gamed. Maybe it's a patient satisfaction score that's high because the surveys are being filtered.
In John Shook's class that I took on A3 problem solving, he pushed students to wrestle with two questions:
- What do you know?
- And how do you know it?
If you think production is hitting target, how do you know? Because the report says so? That's not good enough. Have you watched the work?
One of my favorite GM war stories from the mid-1990s makes the point. A plant production superintendent was fooled by perpetually faked data. He trusted the report. The actual production rate rarely matched the consistent 60 engine blocks per hour the data showed. I wrote about this in GM Got Gamed. The superintendent never went to the gemba in a meaningful way, so he never saw the gap between what the data said and what the work was doing.
Going to the Gemba
Going to the gemba isn't tourism. It isn't a walkthrough where leaders nod and leave. It's the active practice of comparing what the reports say with what the work actually is. That comparison is where learning happens – and where data quality problems get exposed.
A few practical questions leaders can ask:
- Where does this number come from? Who enters it, and when?
- What happens to a person who reports a bad number honestly?
- Have we ever traced a metric back to the source to confirm it's measuring what we think it's measuring?
- When the data looks impossibly good, do we celebrate it or get curious about why?
One commenter on the original version of this post put it well: a benefit of a Lean implementation that often goes unnoticed is the discovery of how good – or how poor – the organization's data actually is. Many organizations are uncomfortable with what they find when they compare their electronic data to reality.
That discomfort is the point. It's where the work starts.
FAQ
Did Deming actually say “In God we trust, all others bring data”?
The quote is widely attributed to him, and he used it often, but he probably didn't originate it. Earlier versions of the saying appear in statistical and business literature before Deming popularized it. The attribution matters less than the underlying discipline he taught: make decisions based on evidence, not authority or opinion.
What did Taiichi Ohno mean by “facts” versus “data”?
Ohno valued data but didn't trust it as a substitute for direct observation. Data is filtered, summarized, and sometimes manipulated by the time it reaches a manager. Facts – what's actually happening at the workplace – are accessible only by being there. The Toyota Production System builds in this practice through gemba walks, andon signals, and direct involvement of leaders in the work.
How can leaders tell when their data is misleading them?
Look for signals like: numbers that never vary (real processes always have some variation), metrics that always hit target (suspicious in any complex system), and a culture where reporting bad news gets punished. Process Behavior Charts help separate signal from noise so leaders can stop reacting to every blip. Charts alone won't fix bad data, though – that takes gemba presence and psychological safety so people feel safe reporting reality.
Do things like this happen in your organization? What did it take to see it?
I agree about the crucial difference between data and facts. But that leads me to ask…
– How are facts stored in an organization?
– How are facts made available to others?
It can’t be the case that every fact must be confirmed by personal observation or we’d spend all our time in the observation stage. I think that data quality and reliability are essential to any organization (lean or not). This is the unfortunate Achilles heal of many organizations and leads to decision-making based on a fuzzy collection of subjective assessments rather than on facts.
One often-missed benefit of a lean implementation is discovery of how good the data is in an organization and especially the data that’s been the basis of business decision-making all along. I’ve worked with a lot of companies who were pretty uncomfortable with what they discovered when they compared their electronic data to reality.
Data is a huge issue. Thanks for bringing this one up.
Very good point! But, you know, data is usually incorrect either because we entered it incorrectly or we managed it badly.
We are responsible for our data (especially the one we create) and there are no machines and/or systems (yet) that can manage it for us. Data integration tools are very important and can help a lot, but we’re the ones building and configuring them.
At the onset of their lean journey, people often feel overwhelmed by the breadth and depth of the various lean principles, systems and tools (I know that I did). I tell them that they will develop in their understanding as they study, see and do, but perhaps the single most important thing that they can bring (along with intellectual curiosity, passion, respect, humility, etc.) is common sense. Most people think they have that! Common sense, “native good judgment,” is typically unvarnished and unsophisticated. There’s nothing as unsophisticated or powerful as going to the gemba and observing the reality or facts before you. So, here’s to common sense and facts.
.-= Mark R Hamel ´s last blog ..The Kaizen Promotion Office Does What? =-.
Hi,
your mistake, though, is to confuse data with information. In the above example, the display showing 119 degrees is a source of information. Data for that day, from NOAA for example, would give us the correct temperature and expose the misinformation.
[…] Toyota’s Taiichi Ohno also famously said: […]
[…] Organizations often worship at the altar of being “data driven.” As Toyota’s Taiichi Ohno once said: […]
[…] I can bet the organization won’t be very strong. Facts and data are not the same thing, as Mark Graban commented on. Any time a measurement is tied to an incentive, then it is likely to be manipulated at some level. […]
Once deviations have turned to data, it’s probable that the wisdom of the moment and people around it have been lost, and it is very difficult to get back to the rootcause. Maybe the difference is between schools of thought, Six Sigma and Lean. Do you want to ride with Motorola and GE or Toyota? Both have their time and place.
What do you think?
GE, in its current incarnations, isn’t really a Six Sigma company anymore. They are all in on Lean:
https://www.leanblog.org/2024/03/safety-and-lean-flight-deck-featured-in-ge-aerospace-investor-day-2024/