Dear Rob and Jack
This is a terrific debate, entertaining, illuminating, instructive
and just the sort thing to warm us up in the morning over here in
foggy England.
The reason that I find the exchanges so useful is that it helps me to
comprehend the way that people, very intelligent people, can and do
conceptualise statistical issues.
Most engineers, chemists, managers, social scientists and the like
that I meet do have some difficulty with these concepts.
The reason that the Normal dist' is so useful is that arithmetic
averages of measurements from any distribution that is
observed, will themselves be approximately Normally dist' provided
that the number of items from which the average is calculated is
big enough. So, for example, a persons average 'score' of some
measurement over a period of time, say X, (averaged over several, say
n, measurements) will behave as a 'Normal' dist provided n is big
enough. We could plot a bargraph of many Xs and the bell shape will
appear, provided no change has occurred over time.
So all distributions 'gravitate' or 'evolve' to 'Normal' in that
sense.
We could also plot the individual Xs on a time plot, and if the Xs
go up, it may indicate improvement in X over time. We can tell if
this improvement is statistically significant. We can take into
account assessor error, we can take into account the reliability of
the scoring system. Similarly we can systematically monitor the
assessors performance, compare it to others, itself, benchmark it.
In fact if you can measure something we can find a statistically
valid way to monitor it over time. We don't actually have to use the
Normal dist' for this.
The fundamental issue seems to be that of measuring differences,
either between some average or some target, and the individual or
group results over time. This in the face of measurement error
(assessor variance). The issues of the measurement score itself ,
and it's frequency, are also critical.
Why don't you get together to try out a few investigations?
The end product could be a way to prove incorrect use of the bell
curve when it happens, and provide ways of finding bias
in assessments and procedures, valid ways of spotting 'good' and
'bad' performance, 'good' and 'bad' schemes etc.
Bearing in mind your opposite viewpoints, this would undoubtedly be
rigorous and could be very useful for the whole field.
Regards
Dave Stewardson
ISRU {Industrial Statistics Research Unit}
MMME {Department of Mechanical, Materials and Manufacturing Engineering}
Stephenson Building
University of Newcastle upon Tyne
Tyne & Wear
England
GB - NE1 7RU
TEL 00 44 191 222 6244
FAX 00 44 191 222 7365