Discussion: View Thread

Gentle Readers:

You gotta admire John Naman's explanation. (Really well done, huh?)

John's explanation reminds me of some research I read a number of years
ago which came out of the reading center in Champaign-Urbana, Illinois.
Essentially, the research questioned how medical students best learned
complex models of the human body. For example, the operation of a muscle
can be explained as a pump, as a rowing team, and with a number of other
metaphors equally well. Metaphors help students early on to grasp
relationships and distinctions, but no metaphor explains the complexity of a
medical dynamic. (Physically, we are quite complex.) So the problem for
medical schools is, how to best teach principles and models without having
students glom-onto metaphors fixedly? The best answer, the researchers
found, was to present various metaphors one at a time, force students to
apply them, and then to move on to the next metaphor *before the students'
conceptualizations were frozen* onto one metaphor. In other words, move
very quickly through various metaphors, just long enough for students to get
the idea--then move on! The researchers also suggested that knowledge of
ill-structured knowledge domains should not be taught sequentially, but
randomly. The reason? The researchers argued that the complexity of
knowledge about ill-structured domains is so great that *no articulation*
would suffice. Hence, the best knowledge representations are those that are
formed in students' head, for those representations are much too complex
(and dynamic) to articulate by traditional means (i.e., semantic or
procedural knowledge).

Before I forget, the articles / papers are:

Spiro, R., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. 1988.
Cognitive flexibility theory: advanced knowledge acquisition in
ill-structured domains. In Tenth Annual Conference of the Cognitive Science
Society. Hillsdale, NJ: Erlbaum.

Spiro, R., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. 1989. Multiple
analogies for complex concepts: antidotes for analogy-induced
misconceptions in advanced knowledge acquisition. In S. Vosniadou & A.
Ortony (Eds.), Similarity and Analogical Reasoning. Cambridge: Cambridge
University Press.

Spiro, R., W. Vispoel, J.Schmitz, A. Samarapungavan, & A. Boerger. 1987.
Knowledge acquisition for application: cognitive flexibility and transfer
in complex content domains. In B.C. Britton (Ed.), Executive Control
Processes. Hillsdale, NJ: Erlbaum.

This research would seem to suggest additional distinctions about
layered cognitive processes (ala John Naman) and how we teach what we teach.
In one sense, we should probably teach simpler things first and more complex
things later one. In another sense, we should not necessarily teach the
simpler things simply or simple-mindedly. In this last regard, both Rausch
and Boal have made contributions here: i.e., the use of questioning, the
emphasis on experience, making criteria explicit, recognizing
cultural-cognitive limitations, and the use of imagination. (I have some
comments about what has seemed to me to be mechanistic procedures for
"correct" problem solving, but later to that.)

I don't know exactly how the rest of you think about these things, but
after practical and academic experience, I've come to think that some of the
stuff we teach is really complex and really hard to get right. At times
general management seems so complex and dynamic that I stand speechless and
amazed in my own classroom. (I mean, if it were so darned simple, then how
come I ain't rich?)

M.

Michael Levenhagen
High Technology Strategy & General Management
College of Business, California Polytechnic
San Luis Obispo, CA 93407
805-756-1563 (off), 756-1473 (fax)
408-456-0704 (hm)

As always, the contributions of folks, like Mike Levenhagen and John Naman,
enrich our appreciation of the difficult tasks of managing and teaching.

Building upon their comments, let me suggest two additional reasons why
both tasks are so complex and difficult.

1) People differ not only with respect to their use of information, but
also with respect to the degree to which they actively seek it. These
dimensions are independent of each other.

2) People differ not only with respect to cognitive complexity, but with
respect to behavioral complexity as well. These dimensions are independent
of each other.

Regarding the first point, Carlton Whitehead and I have suggested that
people range from high to low on both dimensions. People who are low are
both, we label information avoiders. People who are high on both, we label
information junkies. Those who seek information, but don't use it, we
label informaion discarders, and finally, those who will use information,
but don't actively seek it, we label information sensitives.

Regarding the second point, people differ in the range and scope of their
cognitive schemas, as well as the range and scope of behavioral response
repertoires. Those high on both, we label informed flexibles, those low on
both, we label plodders. Those who possess many cognitive schemas, but
limited behavioral repertoires, we label programmed. Finally, those with
few schemas, but many behavioral responses, we label scatter shooters.

For those interested in a discussion of these ideas, I refer you to:

Boal, Kimberly B., and Whitehead, Carlton J. 1992. A critique and
extension of stratified systems theory. In R. L. Phillips and James B.
Hunts (eds), Strategic leadership: A multiorganizational-level
perspective: 237-253. Quorum Books.

At 03:50 PM 12/2/98 -0800, you wrote:
> Gentle Readers:
>
> You gotta admire John Naman's explanation. (Really well done, huh?)
>
> John's explanation reminds me of some research I read a number of years
>ago which came out of the reading center in Champaign-Urbana, Illinois.
>Essentially, the research questioned how medical students best learned
>complex models of the human body. For example, the operation of a muscle
>can be explained as a pump, as a rowing team, and with a number of other
>metaphors equally well. Metaphors help students early on to grasp
>relationships and distinctions, but no metaphor explains the complexity of a
>medical dynamic. (Physically, we are quite complex.) So the problem for
>medical schools is, how to best teach principles and models without having
>students glom-onto metaphors fixedly? The best answer, the researchers
>found, was to present various metaphors one at a time, force students to
>apply them, and then to move on to the next metaphor *before the students'
>conceptualizations were frozen* onto one metaphor. In other words, move
>very quickly through various metaphors, just long enough for students to get
>the idea--then move on! The researchers also suggested that knowledge of
>ill-structured knowledge domains should not be taught sequentially, but
>randomly. The reason? The researchers argued that the complexity of
>knowledge about ill-structured domains is so great that *no articulation*
>would suffice. Hence, the best knowledge representations are those that are
>formed in students' head, for those representations are much too complex
>(and dynamic) to articulate by traditional means (i.e., semantic or
>procedural knowledge).
>
>Before I forget, the articles / papers are:
>
>Spiro, R., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. 1988.
>Cognitive flexibility theory: advanced knowledge acquisition in
>ill-structured domains. In Tenth Annual Conference of the Cognitive Science
>Society. Hillsdale, NJ: Erlbaum.
>
>Spiro, R., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. 1989. Multiple
>analogies for complex concepts: antidotes for analogy-induced
>misconceptions in advanced knowledge acquisition. In S. Vosniadou & A.
>Ortony (Eds.), Similarity and Analogical Reasoning. Cambridge: Cambridge
>University Press.
>
>Spiro, R., W. Vispoel, J.Schmitz, A. Samarapungavan, & A. Boerger. 1987.
>Knowledge acquisition for application: cognitive flexibility and transfer
>in complex content domains. In B.C. Britton (Ed.), Executive Control
>Processes. Hillsdale, NJ: Erlbaum.
>
> This research would seem to suggest additional distinctions about
>layered cognitive processes (ala John Naman) and how we teach what we teach.
>In one sense, we should probably teach simpler things first and more complex
>things later one. In another sense, we should not necessarily teach the
>simpler things simply or simple-mindedly. In this last regard, both Rausch
>and Boal have made contributions here: i.e., the use of questioning, the
>emphasis on experience, making criteria explicit, recognizing
>cultural-cognitive limitations, and the use of imagination. (I have some
>comments about what has seemed to me to be mechanistic procedures for
>"correct" problem solving, but later to that.)
>
> I don't know exactly how the rest of you think about these things, but
>after practical and academic experience, I've come to think that some of the
>stuff we teach is really complex and really hard to get right. At times
>general management seems so complex and dynamic that I stand speechless and
>amazed in my own classroom. (I mean, if it were so darned simple, then how
>come I ain't rich?)
>
>M.
>
>Michael Levenhagen
>High Technology Strategy & General Management
>College of Business, California Polytechnic
>San Luis Obispo, CA 93407
>805-756-1563 (off), 756-1473 (fax)
>408-456-0704 (hm)
>
--------------------------------
Kim Boal
College of Business Administration
Texas Tech University
Lubbock, TX 79409
(806) 742-2150
KimBoal@ttu.edu

Levenhagen's description of the mental models (Gentner & Stevens) that
medical students use reminds me of a funny paper that can be very useful
for managerial education. The paper relates a story approximately as follows:
A prominent medical school screens applicants by MCAT and then interviews
candidates on campus. The most "suitable" candidates are accepted and the
rest rejected. However, very late in the season the state came up with extra
money and wanted more docs, so the school made offers to the rejects.
The "best" of the rejects had gone to other med. schools, so they really got
the tail of the distribution. The clever authors tracked the class's progress
in this "natural" experiment and found no difference in graduating GPAs and
no difference in in evaluations as interns or residents. Moral: the campus
interviews were a waste of time for both management (profs) and interviewees
(students).

Most managers and doctors find this study very hard to swallow, although they will
admit that they have never admited/hired people without all the interviewing.

Source: DeVaul, Richard A., MD, Jervey, Faith, Chappell, James A., MD,
Caver, Patricia, Short, Barbara, and O'Keefe, Stephen 1987. "Medical School
Performance of Initially Rejected Students", JAMA, 257(1):47-51.

John Naman's comments regarding the predictive validity of the interview
for GPA's should come as no surprise. Some years ago, Hunter & Hunter
reported in their meta-analysis of various predictors of job performance
the following criterion-predictor validities for the interview:

1) When the criterion was supervisor ratings: .14

2) When the criterion was promotion: .08

3) When the criterion was training success: .10

4) When the criterion was tenure: .03

Now these dismal validity coefficients are for interviews in general and
make no attempt to distinguish between well done targeted interviews and
the kind of ad hoc interivews that typify most hiring situations.

Given the ubiquity of the interview as the primary basis for selecting
individuals, I doubt that many people are willing to abandone it.
Therefore, I think we have some obligation to try to improve it. I am sure
that everyone has their own idea of what a good (targeted) interviewing
process would look like. Mine includes the following:

1) The interview is proceeded by a thorough job analysis to determine the
KSAs that are needed to perform the job.

2) Only questions that are relevant to the results from the job analysis
are asked.

3) More than one assessor records their evaluation to the questions. The
responses are averaged.

4) Where possible a concurrent validation study is carried out with
current employees to refine the list of questions asked. In some cases,
where questions are highly KSA specific, and that KSA is required on more
than one job in the organization, then a synthetic validation study may be
appropriate to insure a sufficient sample size for analysis.

5) results from step four are used with future job applicants.

My guess is that none of the preceeding occurred in the Medical School study.

What say you?

Regards, Kim Boal



At 03:09 AM 12/4/98 -0400, you wrote:
>Levenhagen's description of the mental models (Gentner & Stevens) that
>medical students use reminds me of a funny paper that can be very useful
>for managerial education. The paper relates a story approximately as follows:
>A prominent medical school screens applicants by MCAT and then interviews
>candidates on campus. The most "suitable" candidates are accepted and the
>rest rejected. However, very late in the season the state came up with extra
>money and wanted more docs, so the school made offers to the rejects.
>The "best" of the rejects had gone to other med. schools, so they really got
>the tail of the distribution. The clever authors tracked the class's progress
>in this "natural" experiment and found no difference in graduating GPAs and
>no difference in in evaluations as interns or residents. Moral: the campus
>interviews were a waste of time for both management (profs) and interviewees
>(students).
>
>Most managers and doctors find this study very hard to swallow, although
they will
>admit that they have never admited/hired people without all the interviewing.
>
>Source: DeVaul, Richard A., MD, Jervey, Faith, Chappell, James A., MD,
>Caver, Patricia, Short, Barbara, and O'Keefe, Stephen 1987. "Medical School
>Performance of Initially Rejected Students", JAMA, 257(1):47-51.
>
--------------------------------
Kim Boal
College of Business Administration
Texas Tech University
Lubbock, TX 79409
(806) 742-2150
KimBoal@ttu.edu

With all due respect, I disagree with Kim's conclusion that "Given the ubiquity
of the interview as the primary basis for selecting individuals, I doubt that
many people are willing to abandon it. Therefore, I think we have some
obligation to try to improve it."

The cost of improving interviewing is not likely to have a large effect on
predictability. If a technique is flawed to the core, then change it. People
used to measure skull size and shape for intelligence and there is no way to
"fix" the flaw in such a technique. Reading tea leaves doesn't cut it for
forecasting.

One Solution: A great psychologist, Bill Howell, proposed that when a qualified
pool exists for employment, promotion, admittance, etc., one should select
among them randomly (a lottery). The benefits include: less hard feelings
because the process is not personal, zero selection bias (gender, race, etc.),
and some subtle good effects of broadening diversity. To make it work for a
particular position, apply the same analysis that you would have to the
interviewing to be sure that the screen (GMAT, GPA, experience, etc.) for the
pool includes skills most relevant to success in the position.

--
Prof. John L. Naman naman+@pitt.edu