As a frequent flyer on American Airlines and a former DFW-area resident, I pay a lot of attention to the troubled airline. American is currently going through a bankruptcy, er “restructuring,” and there’s a lot of what’s called “labor strife” happening right now. I’m curious why these labor/management disputes are never called “management strife,” but that’s a different topic.
After a judge allowed the airline to kill the union contract with the pilots, the pilots are understandably upset. Back in 2003, the pilots and other employees took pay cuts to help avoid bankruptcy (for which, airline executives gave themselves huge bonuses)… despicable. There have been a lot of delayed flights and cancelations lately… is it due to a pilot “sick out,” as claimed by management? And are there lessons that relate to hospital data and trends?
The pilots union says there is not a sickout happening, saying instead that the airline is understaffed.
What do the data show?
From a blog by an aviation reporter at the Dallas Morning News: “Union says sick leave numbers show that pilots aren’t calling in sick in elevated numbers,” you can see selected data on the percentage of pilots who call out sick on the 18th of each month going back 12 months. If the 18th of the month isn’t always a weekday, the data could be skewed by that, but let’s assume that a pilot has an equally likely chance of being sick on any given day of the week.
The reporter makes two comparisons of two data points, something that leads to faulty conclusions, as the great Donald Wheeler teaches in his seminal book Understanding Variation: The Key to Managing Chaos.
By my calculations, the number of pilots on sick leave was 45.7 percent higher on Sept. 18, 2012, than on Sept. 18, 2011, up 177 pilots, That seems like an increase in sick leave usage.
OK, it’s higher on 9/18/2012 than on 9/18/2011. What does that tell us? Just that it was higher on the one day than it was on the other. It’s hard to find a trend from just two data points. I made a chart showing that comparison:
Of course, you could abuse the data by making the Y-axis scale go from 4% to 8%, making the difference look bigger (which would, of course, be misleading):
You could also look at a two data point comparison of August and September 2012:
That shows an increase… but is there evidence of a sick out?
As the writer says:
[September 2012] is lower than the high point over the past 12 months, October 2011, and in the vicinity of sick leave in recent months.
The company claims the data show evidence of a sick out, but it’s in their interest to cherry pick two data points or claim a trend when it might not be there. My approach would be to use the lessons from Wheeler’s book to draw a simple SPC chart… or at least a run chart showing the 12 month trend, as I’ve done below:
As Wheeler teaches, this chart provides so much more context than just two data points. In my view, this chart is far less compelling in terms of the airline management’s case… I don’t see evidence of a sick out or elevated sick day levels.
Now, if you use Excel to draw a simple linear trend line (as people often do) you see this:
There might be an upward trend, but it’s slight and might not represent a “sick out” – who knows? I’d think a “sick out” would be a large increase in sick days, one large enough that it would really cause chaos in the operations of American Airlines. Were there huge cancelation and delay problems in October 2011 and June 2012? If there’s any month that looks like a “sick out,” if would be October 2011 (it looks a bit like a “special cause” in the parlance of SPC. Maybe the slightly higher percentage of sicknesses is causing big service interruptions because the airline is now understaffed, as the union would claim.
A full blown SPC chart would look like this:
The supposed “sick out” data point (Sept 2012) isn’t anywhere close to the Upper Control Limit (the purple line) that’s calculated from the data points (keep in mind an SPC chart should have about 20 data points to be most valid, so I’d love to see data back to the start of 2011). When looking at some of the “Western Electric Rules” used to evaluate control charts to find evidence of a “special cause” (like a sick out):
- 8 consecutive points above the mean (the red line) — nope
- 8 consecutive increasing data points — nope
- A single point above the purple line — nope
If anything, we have too many data points clustered within +/- one sigma of the mean… that would indicate a non-random process that can’t be predicted, perhaps, with an SPC chart or control chart.
If we remove the first two data points and recalculate the control chart limits, I still don’t see a special cause or evidence of a sickout:
It’s hard to prove this is a “sick out” from the data – I just don’t see it.
I hope American is able to get me to the Northeast Shingo Prize Conference in Worcester, Massachusetts, where I’ll be speaking tomorrow… I hope to see some of you there. I also hope American is able to get me and wife off on our planned vacation… I’ll have guest posts in this spot from October 1 through October 11.
I’ve seen plenty of hospitals using faulty comparisons of two data points to show “trends” that aren’t really there. Are we reducing E.D. length of stay, the number of pressure ulcers, the number of patient falls, etc.? To prove it, you need more than two data points – the number can’t be better than last month or better than last year for it to be proven…
How do you see this data? How would you analyze it?
About LeanBlog.org: Mark Graban is a consultant, author, and speaker in the “lean healthcare” methodology. Mark is author of the Shingo Award-winning books Lean Hospitals and Healthcare Kaizen, as well as the new Executive Guide to Healthcare Kaizen. Mark is also the VP of Innovation and Improvement Services for KaiNexus.