Easier to Game the Numbers than Fix the System – in Education and Healthcare

Dave Wants YouI’ve cited his book before (including this post with an example from The Office)…  Brian Joiner wrote, in his outstanding book  Fourth Generation Management: The New Business Consciousness,  that there are three things that can happen when you have a quota or a target imposed upon people:

  1. Distort the system
  2. Distort the numbers
  3. Improve the system

Sadly, it’s often easier to do the first two than the latter. The news recently brought us sad examples of distorting the numbers (in other words, cheating) in education and healthcare.

You might have heard about the school cheating indictments in Atlanta, as exposed in this New York Times piece:  Ex-Schools Chief in Atlanta Is Indicted in Testing Scandal (annoying registration possibly required).

The indictments allege egregious cheating, including teachers in a room ERASING and correcting incorrect answers on students’ tests. Who wins? The superintendent, Dr. Beverly L. Hall, who gained fame and riches ($500,000 in performance bonuses). Who loses? The teachers who were pressured into doing things that were clearly unethical to save their jobs and the students, who lost out on funding because their scores were artificially high (not to mention the kids already losing for getting a bad education).

From the article

[Teacher] Ms. Parks admitted to Mr. Hyde that she was one of seven teachers — nicknamed “the chosen” — who sat in a locked windowless room every afternoon during the week of state testing, raising students’ scores by erasing wrong answers and making them right. She then agreed to wear a hidden electronic wire to school, and for weeks she secretly recorded the conversations of her fellow teachers for [prosecutor] Mr. Hyde.

This is the problem with targets… people under pressure will “do what it takes” when fear rules. I’d like to think each teacher would be a  conscientious whistleblower, but I’m going going to blame somebody who needs to keep a job in a bad economy. The blame starts with the superintendent. I wouldn’t indict teachers for being subjected to the tyranny of bad leadership. Dr. Hall previously denied any involvement, but the indictment accuses her of being the mastermind:

Paul L. Howard Jr., the district attorney, said that under Dr. Hall’s leadership, there was “a single-minded purpose, and that is to cheat.”

“She is a full participant in that conspiracy,” he said. “Without her, this conspiracy could not have taken place, particularly in the degree it took place.”

Dr. Hall was also accused of being a bully:

She said teachers were under constant pressure from principals who feared they would be fired if they did not meet the testing targets set by the superintendent.

Dr. Hall was known to rule by fear. She gave principals three years to meet their testing goals. Few did; in her decade as superintendent, she replaced 90 percent of the principals.

Teachers and principals whose students had high test scores received tenure and thousands of dollars in performance bonuses. Otherwise, as one teacher explained, it was “low score out the door.”

There are rumors and allegations of similar cheating in other cities. Sad… but not unexpected given the reliance on scores and rewards.

I have fears of similar dysfunctions occurring in healthcare, as incentives and financial punishments are being used more against doctors and hospitals. If nurse managers are told by a CEO, “get patient falls down to zero or else,” people might game the system or fudge the numbers (although there’s more evidence of a fall and a broken hip… and it’s a “wrong answer” that can’t be erased like a standardized test form could be.

It’s one thing to set a goal (like zero infections) that inspires people to improve their performance. It’s another thing to set a target or quota with the fear of punishment. You’ll get your number… but at what cost?

Brian Maskell sent me a story about a recent scandal in England’s National Health Service (NHS). I’ve written before about distorting the system to meet the A&E four-hour waiting target. Hospitals would keep patients outside (keeping the 4-hour clock from starting) or admit them unnecessarily after 3:55 (to avoid being dinged on the target).

This still seems to be a problem:  NHS still obsessed with waiting time targets, says ex-A&E tsar.

Brian blogged about this here. As he wrote, in part:

Two senior nurses working in a British “Accident & Emergency (A&E)” clinic were suspended last Friday and are subject to a disciplinary inquiry. They are accused of entering false information into the healthcare facilities documentation systems. According to news reports (Click here to read the entire story in the Sentinel  )     They consistently changed the times of patient care within their A&E department to make it look as if most patients were cared for and discharged in less than four hours, when in fact this was not always true.

Each of us has to take responsibility for our actions… but I wouldn’t blame the nurses if they are given unachievable targets (unachievable in the system in which they worked). Quality starts at the top. If you’re going to blame or indict anybody, it should be those who created the system, not those who are behaving rationally and understandably (even if it seems unethical from our perspective on the outside).

Leaders need to take  responsibility  for the consequences of their incentive and target systems.


Thanks for reading! I’d love to hear your thoughts. Please scroll down to post a comment. Click here to receive posts via email.


Now Available – The updated, expanded, and revised 3rd Edition of Mark Graban’s Shingo Research Award-Winning Book Lean Hospitals: Improving Quality, Patient Safety, and Employee Engagement. You can buy the book today, including signed copies from the author.

Related Posts Plugin for WordPress, Blogger...
Please consider leaving a comment or sharing this post via social media.

Mark Graban's passion is creating a better, safer, more cost effective healthcare system for patients and better workplaces for all. Mark is a consultant, author, and speaker in the "Lean healthcare" methodology. He is author of the Shingo Award-winning books Lean Hospitals and Healthcare Kaizen, as well as The Executive Guide to Healthcare Kaizen. His most recent project is an eBook titled Practicing Lean that benefits the Louise H. Batz Patient Safety Foundation, where Mark is a board member. Mark is also the VP of Improvement & Innovation Services for the technology company KaiNexus.

2 Comments on "Easier to Game the Numbers than Fix the System – in Education and Healthcare"

Trackback | Comments RSS Feed

  1. Mark Graban
    Twitter:
    says:

    Misguided on top of stupid – now they are looking to make criminals out of those who fudged the numbers in the NHS.

    LINK

    Nurses have also alleged privately that they have been told to “massage” waiting time figures by changing the recorded time when patients are treated or discharged.

    A recent survey of nurses also uncovered evidence of even more blatant attempts to manipulate information. Four in 10 nurses said they were aware of attempts to change data, and one in 10 saying they had to change times of patient discharge to meet waiting time targets.

    I wouldn’t jail nurses or even mid-level managers who were under the pressure of the tyranny of targets. When they had unrealistic expectations and their jobs were on the line, they acted quite rationally in fudging the numbers.

  2. Mark,
    So much here in this post:
    1) The bullying – Just like the Rutgers coach this week…
    2) Process (driver)measures (test scores, admitting patients after 3:55) should relate to some outcome of better health, or in the case of students, performance at college level after testing.
    3) And then, there is the whole flawed ‘pay for performance’ issue…

    Thanks for sharing.
    John

Post a Comment