College Football Annual Review Problems
The above article talks about how, for all the talk of “computers” selecting the top two Division 1 college football teams for the championship game, most of the inputs to the system (at least two thirds) are from subjective voter rankings (list the top 25 teams in order).
Many of these voters are pretty uninformed about the rankings, as a coach, for example, hasn't seen every team in the country play. They are ranking teams based on scores (which don't always tell the whole story), or they are giving in to reputation or group-think mentality (“I haven't seen Florida play, but everyone else says they are good.”) Or, voters manipulate the system to get the outcome they want.
The BCS rankings look like a highly quantitative system (on the surface) with scores for each team out to three decimal places. Before yesterday's game, Ohio State had a score of “.982” while Michigan had a score of “.943”. Very quantitative… but mostly subjective. Quantitative does NOT equal objective.
As we move into the annual employee ranking and performance review season, are you fooling yourself about how your quantitative employee scoring system is also objective? You might be assigning scores and numbers to employees, but how much of that score is based on subjective supervisor input, politics, and other such maneuvering?
In business, our objective shouldn't be to determine which employee is “best” or “top 10.” Even if we have highly quantitative methods (such as employee sales figures), how much of that score was truly due to that individual's performance and that individual's performance only?
Which is the biggest waste of time, ranking football teams or ranking employees?
A question for the lean companies out there — how does your company incorporate the Deming notions of getting rid of annual performance reviews? Here are some great resources and links on the topic over the Curious Cat website.
As Dr. Deming said:
Evaluation of performance, merit rating, or annual review… The idea of a merit rating is alluring. the sound of the words captivates the imagination: pay for what you get; get what you pay for; motivate people to do their best, for their own good. The effect is exactly the opposite of what the words promise.