Discussion: View Thread

You folks have been talking about a pet peeve of mine for some time -- that
is, forcing a normal distribution on a group of employees for the purposes
of appraisal. The fallacy of this approach is that if your employment
process is doing a good job of identifying, hiring, and retaining quality
employees, and they are performing to their potential, the forced
distribution process will mandate that many appraisals will not reflect
employees' actual performance levels. Most likely, employees will be rated
LOWER than deserved because the hallowed normal distribution will only
permit a small percentage of high merit decisions. When struggling to fit
their ratings to the normal distribution, managers are forced to make
distinctions that do not really exist. Ultimately, this imposition results
in a process that is LOWER in validity (job relatedness) because of the
error of measurement introduced by creating illusory differences.
Ed Kemery
University of Baltimore


>-----Original Message-----
>From: Tim Edlund <tedlund@MORGAN.EDU>
>To: MG-ED-DV@MAELSTROM.STJOHNS.EDU <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
>Date: Tuesday, May 02, 2000 12:32 PM
>Subject: Re: NBA Player Heights
>
>
>This seems clear. It leads to the assumption that a larger proportion of
>"our" employees fall outside the 2 SD range than is actually the case. IF
>we have a forced distribution of rewards, then more people (assuming
>sufficiently large numbers, which is rarely the case) will get the highest
>rewards than deserve them, and more will get the lowest rewards - which
>may be no rewards or even negative rewards - such as layoffs, firing, etc.
>
>Which brings up another problem - if we use forced distributions to
>de-select those at the bottom, then those people will no longer be
>on-board to occupy the bottom, and some of those previously judged to be
>in the middle range will have to be judged sub-standard the next time the
>exercise is done. For firms forced to frequently retrench, this issue
>will continue to exist.
>
>An argument for unions, I suppose! They tend to insist on some sort of
>non-judgemental rule to determine who has to go - such as last in, first
>out!
>
>Tim Edlund, Morgan State University
>
>On Tue, 2 May 2000, Robert Bacal wrote: [truncated to save space]
>
>> let's see if we can tease out the real life
>> implications of having a distribution that one assumes is normal but
>> isn't.
>>
>> So let's say we have a distribution at work (on performance or
>> ability or whatever) that has 77% of people within 2 SD's (we'll
>> assume it's symetrical for now, that's a test we didn't talk about)
>>
>> The company, however is assuming that the distribution is normal
>> (eg. 68% within the "hump and any other characteristics").
>>
>> If the company was handing out rewards based on the normal curve
>> (which would be inaccurate for them), what kinds of errors (if any),
>> might occur?
>>
>> (I don't know the answer to this -- I should be able to muddle
>> through it, since it's a logic thing, but my brain is fuzzy right now).
>>
>> Anyone?
>>
>> What other errors might be made from assuming a normal curve
>> when one doesn't exist?
>
>

I'm curious: how many MBA programs out there use a forced normal
distribution for grades? I frequently teach at a university in Australia
where I am forced to use a normal distribution for grades....and I thought
MY job was to develop the student to their fullest potential!! Thanks,

Mike Shaner
St. Louis University
----- Original Message -----
From: Dr. Ed Kemery <ekemery@UBMAIL.UBALT.EDU>
To: <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
Sent: Tuesday, May 02, 2000 12:07 PM
Subject: Forced normal distributions and performance appraisals


> You folks have been talking about a pet peeve of mine for some time --
that
> is, forcing a normal distribution on a group of employees for the purposes
> of appraisal. The fallacy of this approach is that if your employment
> process is doing a good job of identifying, hiring, and retaining quality
> employees, and they are performing to their potential, the forced
> distribution process will mandate that many appraisals will not reflect
> employees' actual performance levels. Most likely, employees will be
rated
> LOWER than deserved because the hallowed normal distribution will only
> permit a small percentage of high merit decisions. When struggling to fit
> their ratings to the normal distribution, managers are forced to make
> distinctions that do not really exist. Ultimately, this imposition
results
> in a process that is LOWER in validity (job relatedness) because of the
> error of measurement introduced by creating illusory differences.
> Ed Kemery
> University of Baltimore
>
>
> >-----Original Message-----
> >From: Tim Edlund <tedlund@MORGAN.EDU>
> >To: MG-ED-DV@MAELSTROM.STJOHNS.EDU <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
> >Date: Tuesday, May 02, 2000 12:32 PM
> >Subject: Re: NBA Player Heights
> >
> >
> >This seems clear. It leads to the assumption that a larger proportion of
> >"our" employees fall outside the 2 SD range than is actually the case.
IF
> >we have a forced distribution of rewards, then more people (assuming
> >sufficiently large numbers, which is rarely the case) will get the
highest
> >rewards than deserve them, and more will get the lowest rewards - which
> >may be no rewards or even negative rewards - such as layoffs, firing,
etc.
> >
> >Which brings up another problem - if we use forced distributions to
> >de-select those at the bottom, then those people will no longer be
> >on-board to occupy the bottom, and some of those previously judged to be
> >in the middle range will have to be judged sub-standard the next time the
> >exercise is done. For firms forced to frequently retrench, this issue
> >will continue to exist.
> >
> >An argument for unions, I suppose! They tend to insist on some sort of
> >non-judgemental rule to determine who has to go - such as last in, first
> >out!
> >
> >Tim Edlund, Morgan State University
> >
> >On Tue, 2 May 2000, Robert Bacal wrote: [truncated to save space]
> >
> >> let's see if we can tease out the real life
> >> implications of having a distribution that one assumes is normal but
> >> isn't.
> >>
> >> So let's say we have a distribution at work (on performance or
> >> ability or whatever) that has 77% of people within 2 SD's (we'll
> >> assume it's symetrical for now, that's a test we didn't talk about)
> >>
> >> The company, however is assuming that the distribution is normal
> >> (eg. 68% within the "hump and any other characteristics").
> >>
> >> If the company was handing out rewards based on the normal curve
> >> (which would be inaccurate for them), what kinds of errors (if any),
> >> might occur?
> >>
> >> (I don't know the answer to this -- I should be able to muddle
> >> through it, since it's a logic thing, but my brain is fuzzy right now).
> >>
> >> Anyone?
> >>
> >> What other errors might be made from assuming a normal curve
> >> when one doesn't exist?
> >
> >

In management literature, there is often mention of two opposite
effects, the halo and the horns effects. In theory, your description
fits the horns effect. In theory, using this "normal" distribution,
many more employees should fall to the lower category as you describe.
In reality, the halo effect usually wins out.

IF I recruited, hired, and trained my employees to be the best, then my
population should be skewed in the high range. But since I don't want
to admit that I didn't do a good job hiring and training, I will pretend
that what I have are "above normal" in fact, "high performers". That
way, it makes me look better. And, if I can get them greater raises,
and my salary in % above them, that means more money for me. The high
subjectivity of the process encourages me to rank me employees in the
high ranges.

In addition, if my employees are in fact low performers, it reflects
poorly on me as a manager. But, if I have to tell them they are poor
performers, it usually sets up a conflict situation for me. again, I am
encourager to rank them in the high category. This is not to overlook
those managers who take delight in "kicking ass" and beating up on the
poor performers.

Bob

"Dr. Ed Kemery" wrote:
>
> You folks have been talking about a pet peeve of mine for some time -- that
> is, forcing a normal distribution on a group of employees for the purposes
> of appraisal. The fallacy of this approach is that if your employment
> process is doing a good job of identifying, hiring, and retaining quality
> employees, and they are performing to their potential, the forced
> distribution process will mandate that many appraisals will not reflect
> employees' actual performance levels. Most likely, employees will be rated
> LOWER than deserved because the hallowed normal distribution will only
> permit a small percentage of high merit decisions. When struggling to fit
> their ratings to the normal distribution, managers are forced to make
> distinctions that do not really exist. Ultimately, this imposition results
> in a process that is LOWER in validity (job relatedness) because of the
> error of measurement introduced by creating illusory differences.
> Ed Kemery
> University of Baltimore
>
> >-----Original Message-----
> >From: Tim Edlund <tedlund@MORGAN.EDU>
> >To: MG-ED-DV@MAELSTROM.STJOHNS.EDU <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
> >Date: Tuesday, May 02, 2000 12:32 PM
> >Subject: Re: NBA Player Heights
> >
> >
> >This seems clear. It leads to the assumption that a larger proportion of
> >"our" employees fall outside the 2 SD range than is actually the case. IF
> >we have a forced distribution of rewards, then more people (assuming
> >sufficiently large numbers, which is rarely the case) will get the highest
> >rewards than deserve them, and more will get the lowest rewards - which
> >may be no rewards or even negative rewards - such as layoffs, firing, etc.
> >
> >Which brings up another problem - if we use forced distributions to
> >de-select those at the bottom, then those people will no longer be
> >on-board to occupy the bottom, and some of those previously judged to be
> >in the middle range will have to be judged sub-standard the next time the
> >exercise is done. For firms forced to frequently retrench, this issue
> >will continue to exist.
> >
> >An argument for unions, I suppose! They tend to insist on some sort of
> >non-judgemental rule to determine who has to go - such as last in, first
> >out!
> >
> >Tim Edlund, Morgan State University
> >
> >On Tue, 2 May 2000, Robert Bacal wrote: [truncated to save space]
> >
> >> let's see if we can tease out the real life
> >> implications of having a distribution that one assumes is normal but
> >> isn't.
> >>
> >> So let's say we have a distribution at work (on performance or
> >> ability or whatever) that has 77% of people within 2 SD's (we'll
> >> assume it's symetrical for now, that's a test we didn't talk about)
> >>
> >> The company, however is assuming that the distribution is normal
> >> (eg. 68% within the "hump and any other characteristics").
> >>
> >> If the company was handing out rewards based on the normal curve
> >> (which would be inaccurate for them), what kinds of errors (if any),
> >> might occur?
> >>
> >> (I don't know the answer to this -- I should be able to muddle
> >> through it, since it's a logic thing, but my brain is fuzzy right now).
> >>
> >> Anyone?
> >>
> >> What other errors might be made from assuming a normal curve
> >> when one doesn't exist?
> >
> >

I agree with you 100%. I worked in one of those 'forced distribution'
organizations and doing annual performance appraisals was a real pain
because of the need to distribute the rating accordingly. More often than
not, I would violate the guidance of higher and rate properly, which would
adversely affect my own rating. This problem really became apparent when I
was not in a managerial position with the organization and the site I was in
had to lay someone off. The person selected was a n excellent employee who
had performed well above average for a number of years, but he was the
lowest ranked employee and thus was dumped. I left the organization shortly
after and this person was not considered for re-hire to fill my position
because he was laid off because he was the low performer. That is the real
problem with performance management systems that tie rewards to the ratings.
In order to keep the folks who determine the amount to be given happy, there
must be a way to divvy up the rewards, and someone loses, as Dr, Kemery
says, validity and reliability to a point are adversely affected by this
process and what is worse, if performance management and its appraisal
process are supposed to help management develop employees, the process is
doomed to failure because an employee who is rated accurately and his or her
weaknesses are identified for development purposes will inevitably wind up
on the lower end of the rating scheme and may end being laid off as the low
performer. So, managers inflate appraisals, distribute scores according to
the breakdown determined by higher management and an employees true needs
for development are never identified and documented to assist in the
development process.

Ken Rossi, Ed.D.
Asst. Professor of Information Systems
Hawaii Pacific University
Honolulu, HI 95813
(808) 544-1412
kgrossi@hpu.edu
rossik@hawaii.rr.com
----- Original Message -----
From: "Dr. Ed Kemery" <ekemery@UBMAIL.UBALT.EDU>
To: <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
Sent: Tuesday, May 02, 2000 7:07 AM
Subject: Forced normal distributions and performance appraisals


> You folks have been talking about a pet peeve of mine for some time --
that
> is, forcing a normal distribution on a group of employees for the purposes
> of appraisal. The fallacy of this approach is that if your employment
> process is doing a good job of identifying, hiring, and retaining quality
> employees, and they are performing to their potential, the forced
> distribution process will mandate that many appraisals will not reflect
> employees' actual performance levels. Most likely, employees will be
rated
> LOWER than deserved because the hallowed normal distribution will only
> permit a small percentage of high merit decisions. When struggling to fit
> their ratings to the normal distribution, managers are forced to make
> distinctions that do not really exist. Ultimately, this imposition
results
> in a process that is LOWER in validity (job relatedness) because of the
> error of measurement introduced by creating illusory differences.
> Ed Kemery
> University of Baltimore
>
>
> >-----Original Message-----
> >From: Tim Edlund <tedlund@MORGAN.EDU>
> >To: MG-ED-DV@MAELSTROM.STJOHNS.EDU <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
> >Date: Tuesday, May 02, 2000 12:32 PM
> >Subject: Re: NBA Player Heights
> >
> >
> >This seems clear. It leads to the assumption that a larger proportion of
> >"our" employees fall outside the 2 SD range than is actually the case.
IF
> >we have a forced distribution of rewards, then more people (assuming
> >sufficiently large numbers, which is rarely the case) will get the
highest
> >rewards than deserve them, and more will get the lowest rewards - which
> >may be no rewards or even negative rewards - such as layoffs, firing,
etc.
> >
> >Which brings up another problem - if we use forced distributions to
> >de-select those at the bottom, then those people will no longer be
> >on-board to occupy the bottom, and some of those previously judged to be
> >in the middle range will have to be judged sub-standard the next time the
> >exercise is done. For firms forced to frequently retrench, this issue
> >will continue to exist.
> >
> >An argument for unions, I suppose! They tend to insist on some sort of
> >non-judgemental rule to determine who has to go - such as last in, first
> >out!
> >
> >Tim Edlund, Morgan State University
> >
> >On Tue, 2 May 2000, Robert Bacal wrote: [truncated to save space]
> >
> >> let's see if we can tease out the real life
> >> implications of having a distribution that one assumes is normal but
> >> isn't.
> >>
> >> So let's say we have a distribution at work (on performance or
> >> ability or whatever) that has 77% of people within 2 SD's (we'll
> >> assume it's symetrical for now, that's a test we didn't talk about)
> >>
> >> The company, however is assuming that the distribution is normal
> >> (eg. 68% within the "hump and any other characteristics").
> >>
> >> If the company was handing out rewards based on the normal curve
> >> (which would be inaccurate for them), what kinds of errors (if any),
> >> might occur?
> >>
> >> (I don't know the answer to this -- I should be able to muddle
> >> through it, since it's a logic thing, but my brain is fuzzy right now).
> >>
> >> Anyone?
> >>
> >> What other errors might be made from assuming a normal curve
> >> when one doesn't exist?
> >
> >

Listen to the man! He is sooo right! However, the problem usually devolves
from a less than honest set of appraisals vs. criteria, leaving
organizations with performance rating "inflation", thereby almost forcing a
distribution. (I got all of these ten 4.0 people but number 1 is "more 4.0"
than number 10?) I am reminded of some parallel problems in the Navy, which
at the time I am talking about, forced an order ranking within officer ranks
for regular evaluation occasions, even if the evaluations had identical
scores. Following Ed Kemery's example, this might have fit most units based
on traditional structures. Where it may have hurt (and I say "may" because
I have no evidence to suggest that it did in fact, hurt), was in situations
wherein there were specially selected high-performing people. No matter how
much spin one put on it, a CO was hard-pressed to explain to someone in this
situation why they were ranked where they were. The morale factor could not
have been helped by telling someone that he/she was number 10 of 10 or
worse, 20 out of 20. Likewise, since promotion selection is based on board
action, one can easily see where "rankings" might have worked to the
disadvantage of certain of these individuals, board members not always being
privy to all one would like them to know about individuals and where they
served. I can see its use as an all too convenient and improper
"tie-breaker" at promotion selection times given the factual situation that
some very well-qualified, indeed, outstanding performers cannot be
recommended. And that, of course, takes me back to using
honest/conservative appraisals as careers develop, because if we had high
standards, fairly graded there wouldn't be this push to strive for the curve
almost after the fact as opposed to a curve occurring more normally.
----- Original Message -----
From: "Dr. Ed Kemery" <ekemery@UBMAIL.UBALT.EDU>
To: <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
Sent: Tuesday, May 02, 2000 10:07 AM
Subject: [MG-ED-DV] Forced normal distributions and performance appraisals


| You folks have been talking about a pet peeve of mine for some time --
that
| is, forcing a normal distribution on a group of employees for the purposes
| of appraisal. The fallacy of this approach is that if your employment
| process is doing a good job of identifying, hiring, and retaining quality
| employees, and they are performing to their potential, the forced
| distribution process will mandate that many appraisals will not reflect
| employees' actual performance levels. Most likely, employees will be
rated
| LOWER than deserved because the hallowed normal distribution will only
| permit a small percentage of high merit decisions. When struggling to fit
| their ratings to the normal distribution, managers are forced to make
| distinctions that do not really exist. Ultimately, this imposition
results
| in a process that is LOWER in validity (job relatedness) because of the
| error of measurement introduced by creating illusory differences.
| Ed Kemery
| University of Baltimore
|
|
| >-----Original Message-----
| >From: Tim Edlund <tedlund@MORGAN.EDU>
| >To: MG-ED-DV@MAELSTROM.STJOHNS.EDU <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
| >Date: Tuesday, May 02, 2000 12:32 PM
| >Subject: Re: NBA Player Heights
| >
| >
| >This seems clear. It leads to the assumption that a larger proportion of
| >"our" employees fall outside the 2 SD range than is actually the case.
IF
| >we have a forced distribution of rewards, then more people (assuming
| >sufficiently large numbers, which is rarely the case) will get the
highest
| >rewards than deserve them, and more will get the lowest rewards - which
| >may be no rewards or even negative rewards - such as layoffs, firing,
etc.
| >
| >Which brings up another problem - if we use forced distributions to
| >de-select those at the bottom, then those people will no longer be
| >on-board to occupy the bottom, and some of those previously judged to be
| >in the middle range will have to be judged sub-standard the next time the
| >exercise is done. For firms forced to frequently retrench, this issue
| >will continue to exist.
| >
| >An argument for unions, I suppose! They tend to insist on some sort of
| >non-judgemental rule to determine who has to go - such as last in, first
| >out!
| >
| >Tim Edlund, Morgan State University
| >
| >On Tue, 2 May 2000, Robert Bacal wrote: [truncated to save space]
| >
| >> let's see if we can tease out the real life
| >> implications of having a distribution that one assumes is normal but
| >> isn't.
| >>
| >> So let's say we have a distribution at work (on performance or
| >> ability or whatever) that has 77% of people within 2 SD's (we'll
| >> assume it's symetrical for now, that's a test we didn't talk about)
| >>
| >> The company, however is assuming that the distribution is normal
| >> (eg. 68% within the "hump and any other characteristics").
| >>
| >> If the company was handing out rewards based on the normal curve
| >> (which would be inaccurate for them), what kinds of errors (if any),
| >> might occur?
| >>
| >> (I don't know the answer to this -- I should be able to muddle
| >> through it, since it's a logic thing, but my brain is fuzzy right now).
| >>
| >> Anyone?
| >>
| >> What other errors might be made from assuming a normal curve
| >> when one doesn't exist?
| >
| >
|

Michael - how atrocious! But I think Harvard does the same thing. I tell
my students that I look forward to the day when ALL my students in each
course earn an A. I tell them they all are capable of that - based on the
frequently repeated statement that the best of us uses only 5-10% of our
capacity. The difficulty is, of course, that almost none of us believe
that about ourselves. Yet to increase that use by 2-3% could result is
truly superior results.

I look forward to having to explain to a Dean (should/he ask) why everyone
deserves an A. Unfortunately, I know I will never get that satisfaction.

Tim Edlund, Morgan State University

On Tue, 2 May 2000, Michael Shaner wrote:

> I'm curious: how many MBA programs out there use a forced normal
> distribution for grades? I frequently teach at a university in Australia
> where I am forced to use a normal distribution for grades....and I thought
> MY job was to develop the student to their fullest potential!! Thanks,
>
> Mike Shaner
> St. Louis University

I think Harvard Business School has dropped the forced rankings in the MBA
courses. One of their goals was to develop teamwork skills and by forcing
grading scales where a small percentage would fail regardless of scores was
counter productive. They revamped a bunch of their processes a couple of
years ago.

Ken Rossi, Ed.D.
Asst. Professor of Information Systems
Hawaii Pacific University
Honolulu, HI 95813
(808) 544-1412
kgrossi@hpu.edu
rossik@hawaii.rr.com
----- Original Message -----
From: "Tim Edlund" <tedlund@MORGAN.EDU>
To: <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
Sent: Tuesday, May 02, 2000 10:17 AM
Subject: Re: Forced normal distributions and performance appraisals


> Michael - how atrocious! But I think Harvard does the same thing. I tell
> my students that I look forward to the day when ALL my students in each
> course earn an A. I tell them they all are capable of that - based on the
> frequently repeated statement that the best of us uses only 5-10% of our
> capacity. The difficulty is, of course, that almost none of us believe
> that about ourselves. Yet to increase that use by 2-3% could result is
> truly superior results.
>
> I look forward to having to explain to a Dean (should/he ask) why everyone
> deserves an A. Unfortunately, I know I will never get that satisfaction.
>
> Tim Edlund, Morgan State University
>
> On Tue, 2 May 2000, Michael Shaner wrote:
>
> > I'm curious: how many MBA programs out there use a forced normal
> > distribution for grades? I frequently teach at a university in
Australia
> > where I am forced to use a normal distribution for grades....and I
thought
> > MY job was to develop the student to their fullest potential!! Thanks,
> >
> > Mike Shaner
> > St. Louis University

>When struggling to fit
>their ratings to the normal distribution, managers are forced to make
>distinctions that do not really exist.

Does this remind anyone of Meyer & Rowan's observation that organizations and
their routines are pretty much just "myth and ceremony?" All this stuff about
normal distributions, etc. is just a way to add the appearance of objectivity to
a ceremony around the splitting up of the "spoils" of victory...

Now, I've got to get back to my *grading*...

TR
-------
Terry T. Rock, PhD
Assistant Professor
Faculty of Management
University of Calgary

Tim Edlund wrote (in part)"

>I look forward to having to explain to a Dean (should/he ask) why everyone
>deserves an A. Unfortunately, I know I will never get that satisfaction.

Instead of reams of data, allow me one datum (an anecdote, actually).

Way, way back, a long, long time ago (in 1974), I chaired the awards
committee of what was then known as the National Society for Programmed
Instruction (NSPI). It subsequently became the National Society for
Performance & Instruction and is now known as the International Society for
Performance Improvement (ISPI).

NSPI was (and, as ISPI, still is) heavily populated by people who believe
in criterion-referenced measurement instead of norm-referenced
measurement. So, the year that I headed up the awards committee, we
effected a first: We established in advance the criteria that would be used
to give the awards and wound up giving (as I recall) three awards for
outstanding instructional product and two for outstanding chapter.
Why? Because they satisfied the criteria. Well, the "old guard" had a
fit. They insisted that there was only one "best" instructional product
and only one "best" chapter. However, the society president and vice
president (bless their souls) stood by me (and the principles of the
society) and multiple awards were made. For a couple of years thereafter,
"they" (whomever they might be) returned to the old ways and only one
product and one chapter award was made. But, a few years later, the
criterion-referenced process was made official and they haven't gone back
to the notion of "best" since. There is hope, Tim.

The same is (or could be) true of grades and of performance ratings, that
is, they could be criterion-referenced instead of norm-referenced. But, I
doubt they will be. Here's why:

The performance appraisal system is not a performance appraisal
system at all; that's an artifice, a myth, a fairy tale, a lie. The
performance appraisal system is a management tool and it will be bent to
whatever purpose management happens to have handy. If that's screwing one
group and not another, so be it. If that's figuring out who to fire,
that's the way it will be used. If it's a matter of ensuring that some
people are promoted and others aren't, that is what will happen. The
reason I would like to see norm-referenced performance appraisal systems
scrapped (and I made that clear in my article) is that they aren't
performance appraisal systems at all and they never will be. They are a
management tool and the uses to which that tool are put frequently have
nothing to do with performance; indeed, it could be argued that they rarely
have anything to do with performance.

'Nuff said...

--

Fred Nickols
The Distance Consulting Company
"Assistance at A Distance"
http://home.att.net/~nickols/distance.htm
nickols@worldnet.att.net
(609) 490-0095

Hi,

I think you are refering to the "leniency effect" and the "harshness
effect". The halo/horns effect is when a single favourable/unfavourable
characteristic (to which the appraiser is under a heavy and predominant
influence) tends to "washout" the other (opposite) positive or negative
attributes that might constitute an individual's performance. Although
these performance biases are closely related, and often confused with one
another, they really manifest very different psychological attributions.

Kent Rondeau

At 01:50 PM 05/02/2000 -0400, you wrote:
>In management literature, there is often mention of two opposite
>effects, the halo and the horns effects. In theory, your description
>fits the horns effect. In theory, using this "normal" distribution,
>many more employees should fall to the lower category as you describe.
>In reality, the halo effect usually wins out.
>
>IF I recruited, hired, and trained my employees to be the best, then my
>population should be skewed in the high range. But since I don't want
>to admit that I didn't do a good job hiring and training, I will pretend
>that what I have are "above normal" in fact, "high performers". That
>way, it makes me look better. And, if I can get them greater raises,
>and my salary in % above them, that means more money for me. The high
>subjectivity of the process encourages me to rank me employees in the
>high ranges.
>
>In addition, if my employees are in fact low performers, it reflects
>poorly on me as a manager. But, if I have to tell them they are poor
>performers, it usually sets up a conflict situation for me. again, I am
>encourager to rank them in the high category. This is not to overlook
>those managers who take delight in "kicking ass" and beating up on the
>poor performers.
>
>Bob
>
>"Dr. Ed Kemery" wrote:
>>
>> You folks have been talking about a pet peeve of mine for some time -- that
>> is, forcing a normal distribution on a group of employees for the purposes
>> of appraisal. The fallacy of this approach is that if your employment
>> process is doing a good job of identifying, hiring, and retaining quality
>> employees, and they are performing to their potential, the forced
>> distribution process will mandate that many appraisals will not reflect
>> employees' actual performance levels. Most likely, employees will be rated
>> LOWER than deserved because the hallowed normal distribution will only
>> permit a small percentage of high merit decisions. When struggling to fit
>> their ratings to the normal distribution, managers are forced to make
>> distinctions that do not really exist. Ultimately, this imposition results
>> in a process that is LOWER in validity (job relatedness) because of the
>> error of measurement introduced by creating illusory differences.
>> Ed Kemery
>> University of Baltimore
>>
>> >-----Original Message-----
>> >From: Tim Edlund <tedlund@MORGAN.EDU>
>> >To: MG-ED-DV@MAELSTROM.STJOHNS.EDU <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
>> >Date: Tuesday, May 02, 2000 12:32 PM
>> >Subject: Re: NBA Player Heights
>> >
>> >
>> >This seems clear. It leads to the assumption that a larger proportion of
>> >"our" employees fall outside the 2 SD range than is actually the case. IF
>> >we have a forced distribution of rewards, then more people (assuming
>> >sufficiently large numbers, which is rarely the case) will get the highest
>> >rewards than deserve them, and more will get the lowest rewards - which
>> >may be no rewards or even negative rewards - such as layoffs, firing, etc.
>> >
>> >Which brings up another problem - if we use forced distributions to
>> >de-select those at the bottom, then those people will no longer be
>> >on-board to occupy the bottom, and some of those previously judged to be
>> >in the middle range will have to be judged sub-standard the next time the
>> >exercise is done. For firms forced to frequently retrench, this issue
>> >will continue to exist.
>> >
>> >An argument for unions, I suppose! They tend to insist on some sort of
>> >non-judgemental rule to determine who has to go - such as last in, first
>> >out!
>> >
>> >Tim Edlund, Morgan State University
>> >
>> >On Tue, 2 May 2000, Robert Bacal wrote: [truncated to save space]
>> >
>> >> let's see if we can tease out the real life
>> >> implications of having a distribution that one assumes is normal but
>> >> isn't.
>> >>
>> >> So let's say we have a distribution at work (on performance or
>> >> ability or whatever) that has 77% of people within 2 SD's (we'll
>> >> assume it's symetrical for now, that's a test we didn't talk about)
>> >>
>> >> The company, however is assuming that the distribution is normal
>> >> (eg. 68% within the "hump and any other characteristics").
>> >>
>> >> If the company was handing out rewards based on the normal curve
>> >> (which would be inaccurate for them), what kinds of errors (if any),
>> >> might occur?
>> >>
>> >> (I don't know the answer to this -- I should be able to muddle
>> >> through it, since it's a logic thing, but my brain is fuzzy right now).
>> >>
>> >> Anyone?
>> >>
>> >> What other errors might be made from assuming a normal curve
>> >> when one doesn't exist?
>> >
>> >
>
Kent V. Rondeau, PhD
Assistant Professor and Program Director
Health Policy and Management
Department of Public Health Sciences
University of Alberta
Edmonton, Alberta CANADA T6G 2G3
Telephone: (780)-492-8608
Fax: (780)-492-0364

On 2 May 00, at 18:16, Fred Nickols wrote:


> The performance appraisal system is not a performance appraisal
> system at all; that's an artifice, a myth, a fairy tale, a lie. The
> performance appraisal system is a management tool and it will be bent to
> whatever purpose management happens to have handy. If that's screwing one
> group and not another, so be it. If that's figuring out who to fire,
> that's the way it will be used. If it's a matter of ensuring that some
> people are promoted and others aren't, that is what will happen. The
> reason I would like to see norm-referenced performance appraisal systems
> scrapped (and I made that clear in my article) is that they aren't
> performance appraisal systems at all and they never will be. They are a
> management tool and the uses to which that tool are put frequently have
> nothing to do with performance; indeed, it could be argued that they
> rarely have anything to do with performance.

WELL PUT! Of course the issue REALLY is that since people are
going to continue to use them, can we make the process less of a
lie?

...I think less is possible, as I think Frank Shipper does. How close
to truth is one I'm not sure of.

...but one thing I am sure of. The way to better appraisals is
certainly not through the use of "lying" numbers.


Visit the work911.com supersite at http://www.work911.com
for work related articles, or to find almost anything including
book reviews and suggestions, discussion lists and more.

All three schools that I teach at use criterion based grading. The criteria
is demanding, but achievable. I have yet to have a class of all "A"s ...
but I have had a few exceed-the-standard setting students on almost every
count. I have also had students fail even when the criterion is clear.
And I have had everything between.

I have noticed that "capable of setting a standard" doesn't necessarily
equate to those who do set a standard.

I have personal reasons for being opposed to curves. I grew up in private
schools feeling "less than" because I was not a "top" student until the last
year or so. It was a shock to my system when I went to public college and
discovered that what I thought were skills, capabilities, knowledge and
intelligence were far above the average college student in my area.

I'd much rather set the "beam" and do everything I can to give them the
ability to jump over it. At that point it is their choice.

Conna Condon
DBA Candidate Nova SouthEastern Univ.


----- Original Message -----
From: "Tim Edlund" <tedlund@MORGAN.EDU>
To: <MG-ED-DV@MAELSTROM.STJOHNS.EDU>
Sent: Tuesday, May 02, 2000 1:17 PM
Subject: Re: Forced normal distributions and performance appraisals


> Michael - how atrocious! But I think Harvard does the same thing. I tell
> my students that I look forward to the day when ALL my students in each
> course earn an A. I tell them they all are capable of that - based on the
> frequently repeated statement that the best of us uses only 5-10% of our
> capacity. The difficulty is, of course, that almost none of us believe
> that about ourselves. Yet to increase that use by 2-3% could result is
> truly superior results.
>
> I look forward to having to explain to a Dean (should/he ask) why everyone
> deserves an A. Unfortunately, I know I will never get that satisfaction.
>
> Tim Edlund, Morgan State University
>
> On Tue, 2 May 2000, Michael Shaner wrote:
>
> > I'm curious: how many MBA programs out there use a forced normal
> > distribution for grades? I frequently teach at a university in
Australia
> > where I am forced to use a normal distribution for grades....and I
thought
> > MY job was to develop the student to their fullest potential!! Thanks,
> >
> > Mike Shaner
> > St. Louis University
>

I'd like to thank all of you for participating in this fantastic discussion on
performance appraisals. At times I've laughed, at times I've wondered, but I have
learned lots from the discussion.

I agree with Fred Nickols and Roberta Bacal who have carried much of the load of
this discussion that performance appraisals and the use of normative distribution
systems for rating and rewarding performance contribute very little if anything to
individual performance. Nor do they add to business results (which is what it's
all about). They do however provide a role for human resource professionals, and
often contribute to a feeling of paternalistic "goodness" on the part of senior
leaders. ("Look - see what we are doing for our employees!" And a year later,
employees are laid off with no reference to the performance or the pay for it!) My
personal belief is that they are displacement activities and serve to "displace"
consideration of the much more difficult aspects of building employee performance
and organizational success.

In one organization where I worked, virtually the whole place came to a standstill
because the pay for performance system asked for elements of 360 degree feedback
from other employees, and everyone was so busy writing their appraisals on their
friends and colleagues that their work fell behind!!

In addition, employees can see that those at the high end of the distribution
curve are those who earn the highest amounts anyway. This too leads to feelings of
discontent. Even though performance appraisals and distribution of pay for
performance is supposed to be secret - everyone knows who's getting what and why.

Only once have I seen a reward like this used as an opportunity to build loyalty,
and enhance culture. A senior manager who received quite a large bonus for
performance, came around to the eleven people who worked for him and gave to each
of them a crisp new one hundred dollar bill. That one hundred dollar bill meant
more than all the performance appraisal rewards in the world for the eleven
people. The gesture built loyalty and good will. Unfortuanetly the gesture had to
be kept secret for fear that others (those senior managers who were not so
educated in motivation) would find out and express their displeasure!!

Once people are paid at market rate, performance, in my experience, has very
little to do with pay. It has to do with good will, respect, the opportunity to
participate in challenging work, learning, fairness, trust - all those intagibles
that go to building a positive work and organizational culture. Controlling
through pay (or attempting to) is a cop out and is simply too easy. It may have
worked once in the old days of simple repetitive easily observable tasks, but in
todays knowledge world it doesn't work, and it doesn't contribute to performance.
The unfortunate part of all this is that managers and human resource professionals
all too often retain these schemes and and distribute rewards according to a
normal distribution patterns out of ignorance. They are either too blind to see,
or they know it doesn't work but still do it anyway. We've all got lots of
learning to do in this area.
Jennifer Bowerman,
Edmonton, Alberta

>
>

As I follow the debate on the application of statistical "normalization",
as an epistemology, in understanding performance appraisal phenomena, I
take a deep breath, scratch my head, and attempt to make sense of it
all. Perhaps my leaning toward the application of "critical theory" to
uncovering the depth and richness of "performance appraisal", and other
managerial/organizational phenomena, prevents me from buying into this
obvious application of logical positivism. Critical theory permits me to
investigate the basic organizational assumptions underlying the
application of things like "performance appraisal". Why they exist, what
are their consequences, what are the results of their application, do
they make sense in certain situations and not in others, how do they
affect organizational behavior and or performance?? In any event I hope
that this brief communication is not viewed as that of a "7th grader at
recess":-). On second thought at my age being a 7th grader at recess might
not be a bad thing.

Mike Chumer

In many cases, I have to agree with this assessment. It's unfortunate but I
have seen too many managers totally unaware of what their direct reports
actually do. Hence, they have trouble doing any type of appraisal and resort
to anything that gets them done. In my experience, the basic issue is
writing up sound, reasonable performance criteria understood by both manager
and employee. And this should be a working document. Sometimes jobs evolve
so fast, initial criteria changes so drastically and the appraisal is done
on old material. My two cents.....


>From: Robert Bacal <rbacal@ESCAPE.CA>
>Reply-To: Management Education and Development Discussion
><MG-ED-DV@MAELSTROM.STJOHNS.EDU>
>To: MG-ED-DV@MAELSTROM.STJOHNS.EDU
>Subject: Re: Forced normal distributions and performance
>appraisals
>Date: Tue, 2 May 2000 18:42:39 -0500
>
>On 2 May 00, at 18:16, Fred Nickols wrote:
>
>
> > The performance appraisal system is not a performance appraisal
> > system at all; that's an artifice, a myth, a fairy tale, a lie. The
> > performance appraisal system is a management tool and it will be bent to
> > whatever purpose management happens to have handy. If that's screwing
>one
> > group and not another, so be it. If that's figuring out who to fire,
> > that's the way it will be used. If it's a matter of ensuring that some
> > people are promoted and others aren't, that is what will happen. The
> > reason I would like to see norm-referenced performance appraisal systems
> > scrapped (and I made that clear in my article) is that they aren't
> > performance appraisal systems at all and they never will be. They are a
> > management tool and the uses to which that tool are put frequently have
> > nothing to do with performance; indeed, it could be argued that they
> > rarely have anything to do with performance.
>
>WELL PUT! Of course the issue REALLY is that since people are
>going to continue to use them, can we make the process less of a
>lie?
>
>...I think less is possible, as I think Frank Shipper does. How close
>to truth is one I'm not sure of.
>
>...but one thing I am sure of. The way to better appraisals is
>certainly not through the use of "lying" numbers.
>
>
>Visit the work911.com supersite at http://www.work911.com
>for work related articles, or to find almost anything including
>book reviews and suggestions, discussion lists and more.

________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com

I've only been able to read quickly through the many postings on this subject but would like to suggest that those of you interested in this topic may wish to look at the new work being done by Dr. Meredith Belbin in the area of Work Roles.

Dr. Belbin is widely known for his earlier - and continuing - work in team roles. His new Work Roles is a practical, interactive (360) process for defining the work to be undertaken and for assessing the outcomes of the role. It's specifically designed to address this issue of clarity between supervisor and job holder.

His website is http://www.belbin.com

Regards, Judith Holton



Judith Holton
Strategic Planning & Development
Holland College
140 Weymouth Street
Charlottetown, PE C1A 8N9
Tel. 902 566 9607
Fax 902 566 9549
Email jholton@hollandc.pe.ca
http://www.hollandc.pe.ca

>>> Nancy Cheski <hrdesign@HOTMAIL.COM> 05/03 10:14 am >>>
In many cases, I have to agree with this assessment. It's unfortunate but I
have seen too many managers totally unaware of what their direct reports
actually do. Hence, they have trouble doing any type of appraisal and resort
to anything that gets them done. In my experience, the basic issue is
writing up sound, reasonable performance criteria understood by both manager
and employee. And this should be a working document. Sometimes jobs evolve
so fast, initial criteria changes so drastically and the appraisal is done
on old material. My two cents.....


>From: Robert Bacal <rbacal@ESCAPE.CA>
>Reply-To: Management Education and Development Discussion
><MG-ED-DV@MAELSTROM.STJOHNS.EDU>
>To: MG-ED-DV@MAELSTROM.STJOHNS.EDU
>Subject: Re: Forced normal distributions and performance
>appraisals
>Date: Tue, 2 May 2000 18:42:39 -0500
>
>On 2 May 00, at 18:16, Fred Nickols wrote:
>
>
> > The performance appraisal system is not a performance appraisal
> > system at all; that's an artifice, a myth, a fairy tale, a lie. The
> > performance appraisal system is a management tool and it will be bent to
> > whatever purpose management happens to have handy. If that's screwing
>one
> > group and not another, so be it. If that's figuring out who to fire,
> > that's the way it will be used. If it's a matter of ensuring that some
> > people are promoted and others aren't, that is what will happen. The
> > reason I would like to see norm-referenced performance appraisal systems
> > scrapped (and I made that clear in my article) is that they aren't
> > performance appraisal systems at all and they never will be. They are a
> > management tool and the uses to which that tool are put frequently have
> > nothing to do with performance; indeed, it could be argued that they
> > rarely have anything to do with performance.
>
>WELL PUT! Of course the issue REALLY is that since people are
>going to continue to use them, can we make the process less of a
>lie?
>
>...I think less is possible, as I think Frank Shipper does. How close
>to truth is one I'm not sure of.
>
>...but one thing I am sure of. The way to better appraisals is
>certainly not through the use of "lying" numbers.
>
>
>Visit the work911.com supersite at http://www.work911.com
>for work related articles, or to find almost anything including
>book reviews and suggestions, discussion lists and more.

________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com

Dear List

This all seems like deja-vu.

The numbers don't 'lie', it's they way that they are used which can
be a lie.

The Normal Dist' has been used for over 200 years. For more than the
last 100 years statisticians have been using a multitude of other
models, as and when appropriate. For your task you clearly need a
different model.

From what I have been reading it appears that a certain lack of
numeracy exists, as a rule, within what we used to call the HRM
function. Why try and 'force' a 200 year old model to fit when we
have many others, some that won't need to be 'forced'?

I've said it before, you need to find a friendly local statistician
to help you out.

Regards
Dave Stewardson
ISRU {Industrial Statistics Research Unit}
MMME {Department of Mechanical, Materials and Manufacturing Engineering}
Stephenson Building
University of Newcastle upon Tyne
Tyne & Wear
England
GB - NE1 7RU
TEL 00 44 191 222 6244
FAX 00 44 191 222 7365