web analytics

Gaming the System: Graduation Rates & Pneumonia Rates

Board GamesRegular readers of the blog will know that one of my favorite topics is “Gaming the Numbers” (or “Gaming the System“).

As Brian Joiner so brilliantly wrote, as shared by the late, great Peter Scholtes , there are three things people can do when faced with having to hit goals, targets, or quotas:

(1) improve the system, (2) distort the numbers, or (3) distort the system

Two items were in the news recently about #2 and #3 — from NCAA basketball and from hospitals.

Coach Cal

Kentucky head basketball coach John Calipari seems like the type of coach I’d dislike… a coach who is basically perpetuating a system of “pre-professional” players in a game that’s supposed to be for “student-athletes.”

He was on ESPN recently, plugging his new book and he said some things that actually made me think of Kaizen and Lean leadership. He talked about how important it was to continually practice and to, through the course of a season, figure out the best way to incorporate different players into a system that works. It sounded like a PDSA process where they figure things out experimentally rather than just relying on throwing a bunch of individually talented people into the mix.

But, there was a story that surfaced during the NCAA Tournament that raises an ethical question – is Coach Cal distorting the system and just doing what’s needed to hit a goal set by his organization, or is this dishonest?

As reported by ESPN’s Darren Rovell:

Calipari used to get a $50K bonus if 75% of his senior class graduated. Often had 1 senior. UK deleted clause in 2011. (tweet)

As long as Calipari had a graduation bonus he was incentivized to always have a walk-on senior. Easy $. School realized it. (tweet)

Kentucky is known as a “one and done” school, where basketball players often just play the one season that then allows them to declare for the NBA draft. Most players don’t stay all four years.

I guess 75% of zero wouldn’t count toward that $50,000 bonus. So, the easy way to “game the system” is to invite a walk-on (non scholarship) player onto the team (one you know is going to graduate) and then the graduation rate was one of one or 100% –> bonus!

How’d you like to be that player who realizes that your coach is making $50,000 off of you? I’m glad Kentucky changed that bonus clause.

Gaming the VAP Numbers

I think it’s generally a good trend that CMS and others are no longer paying hospitals for so-called “never events.” It’s a clumsy mechanism, but the dysfunctions caused by the incentive of NOT paying for those preventable cases of patient harm is probably less than the dysfunction of the old system where hospitals got paid for the care provided to treat those preventable errors.

From the WSJ and a letter to the editor (not sure if it’s behind the paywall):

The ACA inserted an economic bias when the incidence of a VAP rate was considered a quality outcome measure and could, if implemented, adversely affect a hospital’s reimbursement rates. Physicians, aware of this have subtly altered the terminology for coding, substituting, for example, a diagnosis of tracheobronchitis for ventilator-associated pneumonia.

As they say, if it’s easier (or perceived as easier) to distort the system than it is to IMPROVE the system… then apparently you get distortion and gaming instead of improvement.


About LeanBlog.org: Mark Graban is a consultant, author, and speaker in the “lean healthcare” methodology. Mark is author of the Shingo Award-winning books Lean Hospitals and Healthcare Kaizen, as well as the new Executive Guide to Healthcare Kaizen. Mark is also the VP of Customer Success for the technology company KaiNexus.

Related Posts Plugin for WordPress, Blogger...
Please consider leaving a comment or sharing this post via social media.

6 Comments on "Gaming the System: Graduation Rates & Pneumonia Rates"

Trackback | Comments RSS Feed

  1. Paul Everett says:

    The creativity of human beings to work something out to their advantage in a recognition system, a bonus system, any kind of data-driven system that directly involves money in one way or another, is limitless. Trying to stop it is a fool’s errand.

    The IRS is another example of such chicanery where 70% of the employees received very large (thousands of tax-payer dollars) “productivity bonuses”, and many of the recipients owed, and had not paid, back taxes. Now that is a sickness that only a wholesale shake up will have any chance of curing.

    Proctor and Gamble’s “Deliberate Change” system, implemented and run by Art Spinanger, sorted it out the best I’ve ever seen. They divided recognition into “Acts of Compensation” and “Acts of Appreciation.” Their focus was on the latter. Every year they had a huge meeting of all the various improvement teams and their projects. What was vied for?? The “Golden Rock” award, which was, literally, a gold-painted rock. I confess I do not know the criteria for receiving the award but I do know it was eagerly contested for.

    P&G no longer has that 30-year successful program because the new leader of it killed it by a major strategic mistake, despite it saving the company, literally, hundreds of millions every year. Same thing happened at a number of other companies with great improvement efforts.

    The dictum “All is change and all is temporary” should be our guide so we won’t be disappointed when our baby is killed.

    Paul

  2. Paul Everett says:

    Mark: I was leader of something called “Better Methods” for many years. Essentially Work Simplification, Material Simplification and Process Improvement. We built a recognition system around the principle: “Creating memories, not money”. Too many details to type but it was incredibly effective at recognizing the great work the foremen did by implementing Better Methods projects that improved productivity and profits in their areas of responsibility. This was from 1966 to 1986 when we expanded the concepts and Lean became the umbrella and involved all the hourly, too.

    The whole effort got killed by a troglodyte VP who prevailed on a weak president to eliminate us in 1992. Such is life. We were in his way. He ruled by fear and didn’t believe in people at all. How he got hired I have no idea. So, a great company builder around Lean was eliminated.

    Today, that same company has a VP charged with doing Lean and he is also head of operations, so maybe the cycle comes around again. He doesn’t really ‘get’ Lean, I don’t think, because he is focused on Kaizen events only. Sigh. I don’t hear about heavy involvement of the hourly in Lean efforts. Big Sigh.

    Paul

  3. Mark Graban
    Twitter:
    says:

    Here is another sad example of “gaming the numbers.”

    There are allegations that 40 vets have died while waiting a long time for appointments at the Phoenix VA.

    Apparently, they are supposed to get appointments within 14 days of a request. But, it was apparently easier to fudge the numbers than to improve the actual system.

    From the WSJ (other articles):

    a VA employee complained to a superior that veterans were being forced to wait some six weeks on a shadow electronic waiting list before being given an actual appointment—in an attempt to make the appointment wait times seem short.

    “Sure, when their appointment is created, it’s [sic] can be 14 days out, but we’re making them wait 6-20 weeks to create that appointment,” said the employee email, dated July 3, 2013. “That is unethical and a disservice to our Veterans.”

    Shameful, but understandable if people are being pressured into a target that they can’t meet. We need to improve the system, not distort the system.
    Mark Graban recently posted..New @KaiNexus Website, New Blog, & Free Improvement WebinarsMy Profile

  4. Laura Montalvo says:

    So true Mark! I have often felt “the pressure” to focus on the “system or score” versus really getting down in the weeds with process and quality improvement projects. Feels like a neverending juggling act…trying to keep my eye on all these moving parts (sometimes in vain) so the tendency is to drop all the balls and focus on the ball that will get reported. Sad, but true. The system of reporting definitely needs to change or gaming will inevitably rear its ugly head again and again!

Post a Comment

CommentLuv badge