Arbitrary Targets at DFW – What About Your Organization?
As I've been re-reading Dr. Deming's Out of the Crisis, stuff like this jumps out at me more and more. I saw similar signs at Heathrow Airport last month bragging about the airport exceeding targets for customer service. This is, of course, the old “comparison to targets” approach to quality that Dr. Deming, Taguchi, and Wheeler have criticized the past few decades.
Quality means more than meeting arbitrary targets set by some bureaucrat.
Here is the big sign that's on the wall in Terminal A at DFW. Click either picture for a larger view.
There are seven metrics tracked and posted for each of the five DFW terminals. What are we supposed to make of this? That they care about customer service and quality?
Let's look at one sign for closer inspection – the ratings for restaurant and eating facilities.
Who sets these arbitrary targets? Who decides that 4.0, or 3.9 to be obnoxiously precise, is good enough? Why do different terminals have different targets? Does Terminal D, with a high volume of international flights, have passengers with higher expectations?
These comparisons are pretty meaningless. Are we supposed to feel good that they are mostly hitting targets? If the target had been set to 4.5 or 3.5, how would we be expected to react? Also, is there anything meaningful here for managers to use in decision making? Probably not! There's probably a lot of back patting taking place because they are hitting targets.
Did the manager of Terminal A lose out on a bonus for missing their target by 0.1? Are the other terminals supposed to stop improving because they hit their target? Wouldn't it be more helpful to talk about WHAT they are doing instead of how they are measuring this? Why is the score low? What would make it higher? Is it a poor choice of restaurants (mostly fast food, except for Terminal D) or poor execution? High prices? I'd feel more assured that quality improvement was coming if they gave at least one cause or reason for the scores not being higher. That might demonstrate some understanding of the system.
As Donald Wheeler writes about in his outstanding book Understanding Variation: The Key to Managing Chaos, we need real process understanding. A single data point, or two data points, has little meaning. A comparison to target might not tell us anything about quality.
If we were going to rely on a measure like this, we would do better to see a run chart showing data or trends over time. A use of Statistical Process Control would be very useful here, as Wheeler teaches.
The SPC methods aren't that complicated to put into place, as I write about in my book, Lean Hospitals: Improving Quality, Patient Safety, and Employee Satisfaction. Instead of comparing patient satisfaction scores to an arbitrary target, track their performance over time in a run chart. Calculate upper and lower control limits that help you keep from over-reacting to every up and down point in the data. This method also helps you keep from over-reacting to every above-average or below-average data point in a stable system. You don't need a master's degree in statistics to do this.
I've helped departments, like hospital laboratories, use this method for looking at turnaround time trends over time – looking for “out of control” data points that indicate a “special cause” worth investigating. Instead of asking “are we hitting our targets?”, a better question is “is our process stable?” What quality would we expect to have next month or next quarter based on past performance? We can't tell from a simple display like this.
If you're new to SPC, maybe I haven't explained this well. Wheeler's book does a great job of teaching this approach, I highly recommend it. I think this whole approach to quality and management is far under-appreciated, even in Lean circles.
What do you think? What do you see in your organization?