Don’t Let Looking at Data Blind You From Seeing Reality


A couple of red traffic lights against a blue skyI recently read an article (a case study) about “Lean Six Sigma” in a publication. It's not online, so I can't link to it, nor do I really want to call them out by name.

I didn't like the article, in part, because it used the old, tired (and wrong) idea that “Six Sigma is for quality” and that Lean is only about “faster and cheaper.”

Good gravy, how do people NOT realize that the Toyota Production System and Lean are about both flow AND quality? For direct evidence that Lean/TPS is about both,  see Toyota's website on “jidoka” and “just in time.”) The article I read is, unfortunately, an example of L.A.M.E. or “Lean As Mistakenly Explained.”

The article talks about a scenario where a city government wants to reduce traffic accidents at one particular intersection. Since an accident is a “defect,” they called on the Six Sigma belts to gather data do a bunch of statistical analysis (assuming, it seems, that a quality problem must be the domain of Six Sigma). It took time and a bunch of meetings to plan for this formal exercise. It's a “hard problem” that's “hard to solve,” the author said.

The Six Sigma team went to the intersection and collected timing and data on the lights and traffic.

After some time, their collected data and analysis showed a problem:

“Anna conducted a cycle-time analysis of the data hoping to uncover patterns that might explain the accidents. Although she didn't find any patterns, she did notice that in several cases, the time deltas were negative. Knowing that cycle-times can't be negative, she at first questioned her team to see if they had entered their data incorrectly. She had each of them switch to a different corner and collect times again. Again, negative durations appeared in the analyzed data. The measurement system was working correctly; something else had to explain the negative times.”

They were, apparently, spending more time looking at the measurement system analysis than the actual traffic.

Why did the data show a problem that they couldn't figure out? The Black Belt finally out and used her eyes. “Go and see,” we say in the Lean/TPS world…

The negative numbers were occurring because, occasionally, one light would turn green about 2 seconds before the cross traffic light turned red. I.e., both directions of traffic were momentarily seeing a Green light and proceeding into the intersection and, thus, colliding with another vehicle.

Read that again. The accidents were happening because, for two seconds, BOTH directions had a GREEN light. Of course there were accidents.

This is the power of Lean problem solving, illustrated (eventually) by a Six Sigma Black Belt:

  • Go to the actual place (get away from the statistic software and go the actual intersection, which they did originally, but not really)
  • Observe with your eyes (really observe carefully and deeply)
  • Talk to the people involved (could they have interviewed some accident victims who might have both said “but I had the green light!”?)

Maybe this is using the benefit of hindsight… but did it really require lots of data collection and time and discussion to discover the source of the problem?

Did this really require Six Sigma? Was this really such a hard problem?

What was the cost of the two weeks spent with the statistical analysis and data collection? “Five serious accidents” occurred in the two weeks people were standing there collecting data instead of really seeing the problem with their own eyes. These are five serious accidents that could have been avoided had this been solved faster.

The author didn't follow his own advice in this case study, it seems:

“…requirement of a good Six Sigma project, that we have a hard problem that others have tried to solve but failed, has been satisfied. If there were a reasonable known solution available, the city should implement it first before considering a Six Sigma project.”

In the case study, the city council says the problem had been around for a long time and they've talked about it… yet they jumped right to the more complex Six Sigma approach.

I guess there are many ways to solve a problem.  I'm not opposed to the statistical methods that are part of Six Sigma.  I'm in general agreement with Toyota when they say it's great to give everybody training in the “seven basic QC tools,” but they don't have formal Six Sigma people or “belts.”

Six Sigma is incredibly helpful for really complicated problems.

But, what I *am* opposed to is inaccurate definitions of Lean that tend to come from “Lean Sigma” folks (like “Lean is just about speed and efficiency”).

I'm also against overly-complicated and slow problem solving methods that relies too much on experts and math when your feet and your eyes would suffice.

It makes me think of the famous Taiichi Ohno quote:

 “Data  is, of course, important in manufacturing,” he often remarked, “but I place greatest emphasis on  facts.”

What do you think? Please scroll down (or click) to post a comment. Or please share the post with your thoughts on LinkedIn – and follow me or connect with me there.

Did you like this post? Make sure you don't miss a post or podcast — Subscribe to get notified about posts via email daily or weekly.

Check out my latest book, The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation:

Get New Posts Sent To You

Select list(s):
Previous articleDenver Bronco Loses Touchdown for Not Following Standardized Work, Dropping Ball Early
Next articlePodcast #185 – Karen Dunn Skinner & David Skinner on Lean in Law Firms and Legal Departments
Mark Graban
Mark Graban is an internationally-recognized consultant, author, and professional speaker, and podcaster with experience in healthcare, manufacturing, and startups. Mark's new book is The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation. He is also the author of Measures of Success: React Less, Lead Better, Improve More, the Shingo Award-winning books Lean Hospitals and Healthcare Kaizen, and the anthology Practicing Lean. Mark is also a Senior Advisor to the technology company KaiNexus.


  1. Nice post Mark. This is one of those where it is hard to tell if the nuance of the problem solving process got lost in the summary or if they were really doing everything they could to hammer through the DMAIC.

    • The thing I was surprised about, reading the case study multiple times, was that they didn’t seem too ashamed about getting that sidetracked on the Six Sigma stuff… it was a very matter-of-fact telling of the story of what happened.

  2. I agree completely. I’m currently struggling at a new organization that’s so tied around Lean Six Sigma that we spend months obsessing with data collection and analysis, instead of simply fixing problems. Many organizations seem to let the “rigor” of Six Sigma be the tail that wags the dog of problem solving.

    The goal is to fix processes and make things work – not to jump through many hoops like those described in your example. Let’s get out there at the Gemba and work to get stuff done.

    • I saw a similar thing at my last manufacturing company, back in 2004, Jason.

      I was working as a Lean coach for one production department that was having trouble meeting customer demand due to shortages of machined components. So, I was working with the department and its internal supplier departments to build up more inventory (short-term countermeasure) to make sure we could then work on the root causes of the inventory shortages (long-term countermeasures). It seemed pretty clear — build up the inventory.

      A colleague was doing a Six Sigma Black Belt certification project with another departments that had parts shortages. She spent MONTHS doing all sorts of statistical analysis and using all of the different Six Sigma tools she needed to use.

      Meanwhile, the customer was not getting its orders. The company was suffering.

      Sometimes, you need to do analysis. Sometimes, you just need to work some overtime to build your inventory levels back up — action over analysis.

  3. It’s also important to note that noticing that the greens overlap is not a root cause. You still have to do root cause analysis to find out why they overlap. Then once you know why, correct it, and observe again to make sure you corrected the situation without causing other issues. Then you have to ask what other intersections have the same root cause and replicate solutions. Accident data might be an important pointer to other intersections with the same issues. Then go and see them!

    • Great point, Steven, that it’s not getting to the root cause yet.

      Sometimes it’s important and expedient to FIX the problem and then go back to try to understand the root cause so, like you said, we can understand why it happened and help prevent it from happening again in other intersections.

      • From Steven on LinkedIn:

        One other point worth mentioning. It’s easy to stereotype six sigma folks as being focused on data and minitab. Six sigma is about using a very structured problem solving approach called DMAIC. Sometimes advanced statistical tools are used (when called for), but often problems are solved easily by going to the process, drawing a simple process map, and examining the way it is failing (failure modes). Some of the best six sigma black belts I know don’t even like stats.

  4. Great analysis, Mark. I believe that incredible results come from the culture change, not deep statistical analysis. At least for a few years. The people who worked for me at a non-profit were not statistically trained, and probably would have found statistics a challenging discipline to learn. Yet they achieved substantial improvements, and sustained them, with lean. Other organizations took the Six Sigma path, and accomplished virtually nothing. It’s hard to explain to someone who has not seen it, which makes going to the Gemba the first thing to do. This does not mean that I don’t find great value in control charts. Understanding common and special cause is very important in determining when and how to act.

    • Yes, control charts are incredibly helpful and I’ve used them as part of my Lean work dating back to my GM days (where we had SPC charts – but not the Deming thinking – before Lean).

  5. Thanks… A great post. I especially appreciated the recognition that certain tools or methods are best for certain kinds of problems (your comment that SS is good at very complicated challenges. Traffic flow and its dynamics are typically a very complex system (read the book “The physics of traffic” for example). But not all problems are complex, or even complicated. The key to optimal response (and probability of successful improvement) lies in knowing what kind of challenge you are dealing with. That, in turn, depends on what is known, knowable, and unknown in the system. In your great story about the intersection, the root cause was readily knowable… If you observed the system. Then, the solution was obvious to all.

  6. Here’s one anecdote I share frequently.

    I got a call from a manufacturing facility that was in a bit of a crisis because their packaging was not staying sealed properly. I was asked to help develop a report to see which assets were producing the most defects.

    I told the quality manager that I’d been to the facility a number of times and had observed these packaging defects moving down the conveyor belt. Not once had I ever seen someone take corrective action.

    I asked him – Do you think a monthly report is going to provide you any more benefit than having the operators, who have full line-of-sight of the conveyor belt, take corrective action as soon as the see the defect? Instead of working on a report, we worked on creating corrective action guides that could be used on the line.

    Don’t get me wrong, I love data analysis. You just to know how and when to use it.

    Before a kaizen workshop, I tend to do as much data analysis as possible. But, I generally withhold all but the top-line results until we start drilling down the chain of causality. Then I’ll pull out the data to check whether or not it supports where we’re going.

    Why not show all the data first? In my experience, it shuts down creativity.

  7. I am reading an interesting book, Culturally on Plan, by Greg Lane. The main focus of the book is the cultural aspects of effective strategy deployment but he devotes a whole chapter on Intuitive vs. Analytical Thinking. He puts forward that intuition is most valuable in stable environments. He cites a multi-year research study where experts made decisions in an unstable environment and did worse than a coin flip. This is where analysis, however slow and painful, is necessary.

    Many world-class manufacturers have been operating in a stable environment for many years. Line-of-sight management can be highly effective in these environments.

    Contrast this with healthcare where we have inherently unstable systems like the ED (show me one that isn’t), or have failed to otherwise provide stability, and line-of-sight decision making doesn’t work.

    (This position also makes the case for why stability is important – it makes it easier to manage)

    It is best to know what kind of road you are driving down when on one side is the ditch of not enough analysis and on the other is too much.

  8. I find it shocking that this could even occur. How about a poke yoke to make it impossible for two greens at the same time? Great post Mark. I really like the simple example shown between the two disciplines.

  9. Agree. As much as I like having hard data to analyze, many times real time observations (gemba facts) are far more enlightening, and data should always be verified by a gemba walk / reality check. Video is also often helpful in understanding and convincing others on what really happens in a process.

  10. This example brought to mind a couple of thoughts. First, it’s another reason why I like to describe Lean Six Sigma as being both an art and a science. It’s an art because different problems/questions require different tools/approaches to resolve/answer them. You have to apprecriate the uniqueness of a given situation and respond to that rather than go in with a pre-planned approach that you will stick with no matter what. Now, if a situation calls for statistical tools, then there is a defined science to those. You don’t get to use a tool designed for comparing means to evaluate variances under the justification of “It’s what I felt like doing.”

    The second thing that came to mind was a quote from a previous Six Sigma trainer that I had – “Remember Jimi Hendrix when you look at a situation – stay as high as you can for as long as you can.”

  11. Hi Mark,

    Re your comment about why the traffic signal control system even allowes simultaneous greens….

    Even with control systems in place, the system may not be designed to function the way most observers would expect. I grew up in the birthplace of the traffic light, Syracuse, New York. In the near Western part of the city, there is a neighborhood that was predominantly Irish-American. At the top of the “Tipperary Hill” in this area, the traffic signal makers at Crouse-Hinds, made a very special light. To honor the Irish heritage there, this light has the green signal on top. It is this way to this day. Good luck to the color-blind. Yes, there have been accidents. But the desire to have the light as it is, has outweighed the desire to change it.

  12. Not being able to read the original case study I can’t say if any of the following supposition is true but I wonder if there was a place to stand where you could see that there were two green lights on at the same time. If not then the only way to “see” that would be to collect the data, see the anomaly in the data and then go back to the junction to see what was happening.

    I’m like you Mark, I’m not such a fan of rigid Six Sigma data analysis, but there are times that the only way to “see” something is through the data. I’ve lost count of number of times that an unexpected seasonality has presented itself after plotting an SPC chart, for example.


Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.