Discussion: View Thread

  • 1.  sorting wheat from chaff across generations

    Posted 10-15-2011 01:37
    Actually my request relates to model and theory development, and hence the capability of using a theory to predict. From my reading and research, tertiary students in a society are a population in and of themselves with their own unique values and value structures, sub-societal culture areas, and religio-ethno-linguistic fractionalisation. Additionally, Shalom Schwartz has commented that students in different majors have different individual value structures.

    If we find that students have different individual value structures from those of a society's businessperson population, then student samples are not acceptable for theoretical models other than for predicting values and behaviour of students. If each generation initiates some sort of change in the values of the population of businesspeople when they enter employment, we need to know that also. Which would take a longitudinal study with a large sample over several decades to be useful.

    Regards,
    Romie Littrell

    P.S. The issue can be much more complex. There are two possible progressions, one, students adopt the values of older businesspeople when they enter employment, or two, the new generation of students begins a process of cultural change in the businesspeople population. If we can identify generational cohorts, such as "Gamers", Generation X and Y, "Baby Boomers" in the USA, or the Communist Consolidation Era, The Great Cultural Revolution Era, and the Social Reform Era in the P.R. China, then we might be able to develop more nearly coherent predictive models. If we cannot, the 20 or 30 separate generations working in business in a society operate in a feedback system, creating a set of population value structures that changes to some degree every year with every generation's graduating class. Of course there are other contingencies impelling change in value structures.

    P.P.S. If we are driven by time, money, and a demand to produce three journal articles per year to get tenure or promotion, then using our convenient student samples may be necessary. However, we should at least know the parameters of the population.
     
    Hope for the USA? "If something is unsustainable, it will stop."--Herb Stein, an economic adviser to Richard Nixon
    Romie F. Littrell, BA, MBA,PhD, FIAIR, An fánaífiáin
    AUT Business School N.Z., romie.littrell@aut.ac.nz
    http://www.romielittrellpubs.homestead.com/
    Facilitator, Leadership & Management in Sub-Sahara Africa Conferences
    Contents copyright Romie F. Littrell
    From: sheila cameron <sheilacx333@GMAIL.COM>
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Sent: Saturday, 15 October 2011, 8:21
    Subject: Re: sorting wheat from chaff

    The gradings were really interesting, but I'm left wondering who, in the case of management research, would do the judging and for whom would they be making recommendations? I thought that the peer review process should have meant that evidence was adequate for the conclusions claimed, though some of the papers I have read do somewhat call this into question. But the greater difficulty is 'useful for whom'? Management research has many different audience 'segments' and ideas might be useful for academics but disastrous if accepted as a prescription for practice. So I'm feeling a bit gloomy about the likelihood of a parallel system for us.... And that is without considering the possibility that some of the most useful research might not fit a positivist 'scientific' paradigm....
    Sheila

    On Fri, Oct 14, 2011 at 7:50 PM, Brown, Kenneth G <kenneth-g-brown@uiowa.edu> wrote:
    Fred,
     
    Thanks, this helps. I wonder if you and others have seen the classification schemes created to "rank evidence."  While there are a variety, here is one:
     
     
    From the US Preventative Services Task Force, this framework includes grades for both "level of certainty" (from high to low, derived from nature of evidence) and "suggestions for practice" (from A to D, or I as insufficient).
     
    What would you and others on the list think about classifying management education findings using such a classification system? This would provide one possible mechanism to help separate wheat from chaff....
     
    Ken
     
    ~~~~~~~~~~~~~
    Kenneth G. Brown, Ph.D., SPHR
    Associate Professor of Management & Organizations and Tippie Research Fellow
    Henry B. Tippie College of Business
    The University of Iowa
    Iowa City, IA 52242
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Fred Nickols
    Sent: Friday, October 14, 2011 1:36 PM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    Ken:
     
    What I take away from the two articles in question is that unless we are capable of authoritatively and competently examining and critiquing the design and conduct of a research effort, we are not in a good position to question – or accept – its findings.  My bottom-line conclusion is that a great deal of so-called scientific research isn't and its findings are not to be trusted.  On the other hand, I'm equally sure there is a great deal of solid scientific research being conducted and reported.  For me, as a non-scientist, the difficulty lies in knowing which is which and, frankly, I'm darned if I can tell.  Even if I were qualified to evaluate research design, conduct and findings, I would hate to have to do that every time an interesting new or contradictory finding was reported.  It seems to me there ought to be some kind of mechanism for sorting the wheat from the chaff.
     
    Regards,
     
    Fred Nickols
    Managing Partner
    Distance Consulting LLC
    Home to "Solution Engineering"
    1558 Coshocton Ave – Suite 303
    Mount Vernon, OH 43050
     
    "We Engineer Solutions to Performance Problems"
     
     
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Brown, Kenneth G
    Sent: Friday, October 14, 2011 10:33 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    Fred, thanks for sending this particular blog link.  I read it over lunch and came away disheartened.  The author (whom I can't identify from the blog) criticized Lehrer for sweeping generalizations but then himself wrote the following:
     
    "The problem isn't that the Decline Effect happens in science; the problem is that we think psychology and ecology and economics are sciences. They can be approached scientifically, but their conclusions cannot be considered valid outside of their immediate context." 
     
    "The problem isn't the Decline Effect, the problem is you believed the data had the force of  F=ma.  No one should be surprised when medical "truths" turn out to be wrong-- they were never true to begin with."
     
    "And the moment you talk to The New Yorker, your misinterpreted statistical association becomes truth."
     
    So I think the blogger here is ranting against the popular press (and scientists who talk to the popular press) and how it reports on findings too early and too flippantly. But he also seems to be dismissing all empirical research and, in particular, dismissing all research excepts physics. 
     
    Some of the studies described by Lehrer are rigourous meta-analyses, synthesized across multiple studies. They are not "n = 4 studies" (to use another quote from the blog post).  In this post I do not see a substantive criticism of the studies that Lehrer discusses as a basis for his views.
     
    As somone interested in the exchange of knowledge between science and practice, I come away thinking that the blog author believes there is no truth and no value in scientists trying to explain their findings to a broader public. This seems to me to be a nihilistic view of all social science, rather than a targeted criticism of the Decline Effect.
     
    Maybe I missed something?
     
    Ken
     
    ~~~~~~~~~~~~~
    Kenneth G. Brown, Ph.D., SPHR
    Associate Professor of Management & Organizations and Tippie Research Fellow
    Henry B. Tippie College of Business
    The University of Iowa
    Iowa City, IA 52242
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Fred Nickols
    Sent: Friday, October 14, 2011 8:13 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    In relation to the article Gary posted, here's one that takes a critical view of the claims made in it.
     
     
     
    Regards,
     
    Fred Nickols
    Managing Partner
    Distance Consulting LLC
    Home to "Solution Engineering"
    1558 Coshocton Ave – Suite 303
    Mount Vernon, OH 43050
     
    "We Engineer Solutions to Performance Problems"
     
     
     
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Gary
    Sent: Thursday, October 13, 2011 6:37 PM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    Colleagues,
     
    You have probably heard about the Decline Effect.
    I seems relevant to Romie's request.
     
    The Decline Effect, New Yorker Magazine.
     
    Disturbing or intriguing.
     
    Best,
     
    Gary Lundquist
     





  • 2.  sorting wheat from chaff across generations

    Posted 10-15-2011 11:28

    Romie Litrell wrote, "If we find that students have different individual value structures from those of a society's businessperson population, then student samples are not acceptable for theoretical models other than for predicting values and behaviour of students."

     

    I agree that student samples can be overused, but I wouldn't go so far as to say they cannot and should not be used for anything other than studying students. If we take this logic to its natural conclusion, then we would never be able to generalize from any single sample to any other sample where differences in values exist (can we generalize from Google employees to Citibank employees, for example). To the degree that values drive the phenomena under study, and are not explicitly measured, generalizability would certainly be a problem. But what about other research questions where values aren't so central?

     

    This issue has been discussed in some detail in prior literature, see for example an exchange between Gordon et al. and Jerald Greenberg. Greenberg's (partial) defense of student samples was this -- "The college sophomore as guinea pig: Setting the record straight" (AMR, 1987).

     

    With that said, there is empirical support for Romie's basic contention that we should be cautious. Although the author does not go so far as to say college students should not be  used, he does find different effect sizes.  See Peterson (2001).  This author posts his work on-line, so its readily available here (http://business.nmsu.edu/~mhyman/M610_Articles/Peterson_JCR_2001.pdf).

     

    Back to work!

     

    Ken

     

     

    Kenneth G. Brown, Ph.D., SPHR

    Associate Professor of Management & Organizations and Tippie Research Fellow

    Editor, Academy of Management Learning & Education

    Henry B. Tippie College of Business

    The University of Iowa

    Iowa City, IA 52242

    Ph: 319.335.3812  Fx: 319.335.1957

     

    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Romie Littrell
    Sent: Saturday, October 15, 2011 12:37 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: sorting wheat from chaff across generations

     

    Actually my request relates to model and theory development, and hence the capability of using a theory to predict. From my reading and research, tertiary students in a society are a population in and of themselves with their own unique values and value structures, sub-societal culture areas, and religio-ethno-linguistic fractionalisation. Additionally, Shalom Schwartz has commented that students in different majors have different individual value structures.

     

    If we find that students have different individual value structures from those of a society's businessperson population, then student samples are not acceptable for theoretical models other than for predicting values and behaviour of students. If each generation initiates some sort of change in the values of the population of businesspeople when they enter employment, we need to know that also. Which would take a longitudinal study with a large sample over several decades to be useful.

     

    Regards,

    Romie Littrell

     

    P.S. The issue can be much more complex. There are two possible progressions, one, students adopt the values of older businesspeople when they enter employment, or two, the new generation of students begins a process of cultural change in the businesspeople population. If we can identify generational cohorts, such as "Gamers", Generation X and Y, "Baby Boomers" in the USA, or the Communist Consolidation Era, The Great Cultural Revolution Era, and the Social Reform Era in the P.R. China, then we might be able to develop more nearly coherent predictive models. If we cannot, the 20 or 30 separate generations working in business in a society operate in a feedback system, creating a set of population value structures that changes to some degree every year with every generation's graduating class. Of course there are other contingencies impelling change in value structures.

     

    P.P.S. If we are driven by time, money, and a demand to produce three journal articles per year to get tenure or promotion, then using our convenient student samples may be necessary. However, we should at least know the parameters of the population.

     

    Hope for the USA? "If something is unsustainable, it will stop."--Herb Stein, an economic adviser to Richard Nixon
    Romie F. Littrell, BA, MBA,PhD, FIAIR, An fánaífiáin
    AUT Business School N.Z., romie.littrell@aut.ac.nz
    http://www.romielittrellpubs.homestead.com/
    Facilitator, Leadership & Management in Sub-Sahara Africa Conferences
    Contents copyright Romie F. Littrell

    From: sheila cameron <sheilacx333@GMAIL.COM>
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Sent: Saturday, 15 October 2011, 8:21
    Subject: Re: sorting wheat from chaff

    The gradings were really interesting, but I'm left wondering who, in the case of management research, would do the judging and for whom would they be making recommendations? I thought that the peer review process should have meant that evidence was adequate for the conclusions claimed, though some of the papers I have read do somewhat call this into question. But the greater difficulty is 'useful for whom'? Management research has many different audience 'segments' and ideas might be useful for academics but disastrous if accepted as a prescription for practice. So I'm feeling a bit gloomy about the likelihood of a parallel system for us.... And that is without considering the possibility that some of the most useful research might not fit a positivist 'scientific' paradigm....

    Sheila

    On Fri, Oct 14, 2011 at 7:50 PM, Brown, Kenneth G <kenneth-g-brown@uiowa.edu> wrote:

    Fred,

     

    Thanks, this helps. I wonder if you and others have seen the classification schemes created to "rank evidence."  While there are a variety, here is one:

     

     

    From the US Preventative Services Task Force, this framework includes grades for both "level of certainty" (from high to low, derived from nature of evidence) and "suggestions for practice" (from A to D, or I as insufficient).

     

    What would you and others on the list think about classifying management education findings using such a classification system? This would provide one possible mechanism to help separate wheat from chaff....

     

    Ken

     

    ~~~~~~~~~~~~~

    Kenneth G. Brown, Ph.D., SPHR

    Associate Professor of Management & Organizations and Tippie Research Fellow

    Henry B. Tippie College of Business

    The University of Iowa

    Iowa City, IA 52242

    Ph: 319.335.3812  Fx: 319.335.1957

     

    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Fred Nickols
    Sent: Friday, October 14, 2011 1:36 PM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample

     

    Ken:

     

    What I take away from the two articles in question is that unless we are capable of authoritatively and competently examining and critiquing the design and conduct of a research effort, we are not in a good position to question – or accept – its findings.  My bottom-line conclusion is that a great deal of so-called scientific research isn't and its findings are not to be trusted.  On the other hand, I'm equally sure there is a great deal of solid scientific research being conducted and reported.  For me, as a non-scientist, the difficulty lies in knowing which is which and, frankly, I'm darned if I can tell.  Even if I were qualified to evaluate research design, conduct and findings, I would hate to have to do that every time an interesting new or contradictory finding was reported.  It seems to me there ought to be some kind of mechanism for sorting the wheat from the chaff.

     

    Regards,

     

    Fred Nickols

    Managing Partner

    Distance Consulting LLC

    Home to "Solution Engineering"

    1558 Coshocton Ave – Suite 303

    Mount Vernon, OH 43050

     

    "We Engineer Solutions to Performance Problems"

     

     

     

    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Brown, Kenneth G
    Sent: Friday, October 14, 2011 10:33 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample

     

    Fred, thanks for sending this particular blog link.  I read it over lunch and came away disheartened.  The author (whom I can't identify from the blog) criticized Lehrer for sweeping generalizations but then himself wrote the following:

     

    "The problem isn't that the Decline Effect happens in science; the problem is that we think psychology and ecology and economics are sciences. They can be approached scientifically, but their conclusions cannot be considered valid outside of their immediate context." 

     

    "The problem isn't the Decline Effect, the problem is you believed the data had the force of  F=ma.  No one should be surprised when medical "truths" turn out to be wrong-- they were never true to begin with."

     

    "And the moment you talk to The New Yorker, your misinterpreted statistical association becomes truth."

     

    So I think the blogger here is ranting against the popular press (and scientists who talk to the popular press) and how it reports on findings too early and too flippantly. But he also seems to be dismissing all empirical research and, in particular, dismissing all research excepts physics. 

     

    Some of the studies described by Lehrer are rigourous meta-analyses, synthesized across multiple studies. They are not "n = 4 studies" (to use another quote from the blog post).  In this post I do not see a substantive criticism of the studies that Lehrer discusses as a basis for his views.

     

    As somone interested in the exchange of knowledge between science and practice, I come away thinking that the blog author believes there is no truth and no value in scientists trying to explain their findings to a broader public. This seems to me to be a nihilistic view of all social science, rather than a targeted criticism of the Decline Effect.

     

    Maybe I missed something?

     

    Ken

     

    ~~~~~~~~~~~~~

    Kenneth G. Brown, Ph.D., SPHR

    Associate Professor of Management & Organizations and Tippie Research Fellow

    Henry B. Tippie College of Business

    The University of Iowa

    Iowa City, IA 52242

    Ph: 319.335.3812  Fx: 319.335.1957

     

    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Fred Nickols
    Sent: Friday, October 14, 2011 8:13 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample

     

    In relation to the article Gary posted, here's one that takes a critical view of the claims made in it.

     

     

     

    Regards,

     

    Fred Nickols

    Managing Partner

    Distance Consulting LLC

    Home to "Solution Engineering"

    1558 Coshocton Ave – Suite 303

    Mount Vernon, OH 43050

     

    "We Engineer Solutions to Performance Problems"

     

     

     

     

    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Gary
    Sent: Thursday, October 13, 2011 6:37 PM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample

     

    Colleagues,

     

    You have probably heard about the Decline Effect.

    I seems relevant to Romie's request.

     

    The Decline Effect, New Yorker Magazine.

     

    Disturbing or intriguing.

     

    Best,

     

    Gary Lundquist

     

     

     



  • 3.  sorting wheat from chaff across generations

    Posted 10-16-2011 12:06
    In College Physics 101 laboratory I learned that every report of experimental findings had to include an 'estimated probable error' with rationale. If ME&D experiments included this then this discussion is moot.

    On Oct 15, 2011, at 8:27 AM, Brown, Kenneth G wrote:

    Romie Litrell wrote, "If we find that students have different individual value structures from those of a society's businessperson population, then student samples are not acceptable for theoretical models other than for predicting values and behaviour of students."
     
    I agree that student samples can be overused, but I wouldn't go so far as to say they cannot and should not be used for anything other than studying students. If we take this logic to its natural conclusion, then we would never be able to generalize from any single sample to any other sample where differences in values exist (can we generalize from Google employees to Citibank employees, for example). To the degree that values drive the phenomena under study, and are not explicitly measured, generalizability would certainly be a problem. But what about other research questions where values aren't so central?
     
    This issue has been discussed in some detail in prior literature, see for example an exchange between Gordon et al. and Jerald Greenberg. Greenberg's (partial) defense of student samples was this -- "The college sophomore as guinea pig: Setting the record straight" (AMR, 1987).
     
    With that said, there is empirical support for Romie's basic contention that we should be cautious. Although the author does not go so far as to say college students should not be  used, he does find different effect sizes.  See Peterson (2001).  This author posts his work on-line, so its readily available here (http://business.nmsu.edu/~mhyman/M610_Articles/Peterson_JCR_2001.pdf).
     
    Back to work!
     
    Ken
     
     
    Kenneth G. Brown, Ph.D., SPHR
    Associate Professor of Management & Organizations and Tippie Research Fellow
    Henry B. Tippie College of Business
    The University of Iowa
    Iowa City, IA 52242
    Ph: 319.335.3812  Fx: 319.335.1957
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Romie Littrell
    Sent: Saturday, October 15, 2011 12:37 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: sorting wheat from chaff across generations
     
    Actually my request relates to model and theory development, and hence the capability of using a theory to predict. From my reading and research, tertiary students in a society are a population in and of themselves with their own unique values and value structures, sub-societal culture areas, and religio-ethno-linguistic fractionalisation. Additionally, Shalom Schwartz has commented that students in different majors have different individual value structures.
     
    If we find that students have different individual value structures from those of a society's businessperson population, then student samples are not acceptable for theoretical models other than for predicting values and behaviour of students. If each generation initiates some sort of change in the values of the population of businesspeople when they enter employment, we need to know that also. Which would take a longitudinal study with a large sample over several decades to be useful.
     
    Regards,
    Romie Littrell
     
    P.S. The issue can be much more complex. There are two possible progressions, one, students adopt the values of older businesspeople when they enter employment, or two, the new generation of students begins a process of cultural change in the businesspeople population. If we can identify generational cohorts, such as "Gamers", Generation X and Y, "Baby Boomers" in the USA, or the Communist Consolidation Era, The Great Cultural Revolution Era, and the Social Reform Era in the P.R. China, then we might be able to develop more nearly coherent predictive models. If we cannot, the 20 or 30 separate generations working in business in a society operate in a feedback system, creating a set of population value structures that changes to some degree every year with every generation's graduating class. Of course there are other contingencies impelling change in value structures.
     
    P.P.S. If we are driven by time, money, and a demand to produce three journal articles per year to get tenure or promotion, then using our convenient student samples may be necessary. However, we should at least know the parameters of the population.
     
    Hope for the USA? "If something is unsustainable, it will stop."--Herb Stein, an economic adviser to Richard Nixon
    Romie F. Littrell, BA, MBA,PhD, FIAIR, An fánaífiáin
    AUT Business School N.Z., romie.littrell@aut.ac.nz 
    http://www.romielittrellpubs.homestead.com/ 
    Facilitator, Leadership & Management in Sub-Sahara Africa Conferences 
    Contents copyright Romie F. Littrell

    From: sheila cameron <sheilacx333@GMAIL.COM>
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Sent: Saturday, 15 October 2011, 8:21
    Subject: Re: sorting wheat from chaff

    The gradings were really interesting, but I'm left wondering who, in the case of management research, would do the judging and for whom would they be making recommendations? I thought that the peer review process should have meant that evidence was adequate for the conclusions claimed, though some of the papers I have read do somewhat call this into question. But the greater difficulty is 'useful for whom'? Management research has many different audience 'segments' and ideas might be useful for academics but disastrous if accepted as a prescription for practice. So I'm feeling a bit gloomy about the likelihood of a parallel system for us.... And that is without considering the possibility that some of the most useful research might not fit a positivist 'scientific' paradigm....

    Sheila

    On Fri, Oct 14, 2011 at 7:50 PM, Brown, Kenneth G <kenneth-g-brown@uiowa.edu> wrote:
    Fred,
     
    Thanks, this helps. I wonder if you and others have seen the classification schemes created to "rank evidence."  While there are a variety, here is one:
     
     
    From the US Preventative Services Task Force, this framework includes grades for both "level of certainty" (from high to low, derived from nature of evidence) and "suggestions for practice" (from A to D, or I as insufficient).
     
    What would you and others on the list think about classifying management education findings using such a classification system? This would provide one possible mechanism to help separate wheat from chaff....
     
    Ken
     
    ~~~~~~~~~~~~~
    Kenneth G. Brown, Ph.D., SPHR
    Associate Professor of Management & Organizations and Tippie Research Fellow
    Henry B. Tippie College of Business
    The University of Iowa
    Iowa City, IA 52242
    Ph: 319.335.3812  Fx: 319.335.1957
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Fred Nickols
    Sent: Friday, October 14, 2011 1:36 PM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    Ken:
     
    What I take away from the two articles in question is that unless we are capable of authoritatively and competently examining and critiquing the design and conduct of a research effort, we are not in a good position to question – or accept – its findings.  My bottom-line conclusion is that a great deal of so-called scientific research isn't and its findings are not to be trusted.  On the other hand, I'm equally sure there is a great deal of solid scientific research being conducted and reported.  For me, as a non-scientist, the difficulty lies in knowing which is which and, frankly, I'm darned if I can tell.  Even if I were qualified to evaluate research design, conduct and findings, I would hate to have to do that every time an interesting new or contradictory finding was reported.  It seems to me there ought to be some kind of mechanism for sorting the wheat from the chaff.
     
    Regards,
     
    Fred Nickols
    Managing Partner
    Distance Consulting LLC
    Home to "Solution Engineering"
    1558 Coshocton Ave – Suite 303
    Mount Vernon, OH 43050
     
    "We Engineer Solutions to Performance Problems"
     
     
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Brown, Kenneth G
    Sent: Friday, October 14, 2011 10:33 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    Fred, thanks for sending this particular blog link.  I read it over lunch and came away disheartened.  The author (whom I can't identify from the blog) criticized Lehrer for sweepinggeneralizations but then himself wrote the following:
     
    "The problem isn't that the Decline Effect happens in science; the problem is that we think psychology and ecology and economics are sciences. They can be approached scientifically, but their conclusions cannot be considered valid outside of their immediate context." 
     
    "The problem isn't the Decline Effect, the problem is you believed the data had the force of  F=ma.  No one should be surprised when medical "truths" turn out to be wrong-- they were never true to begin with."
     
    "And the moment you talk to The New Yorker, your misinterpreted statistical association becomes truth."
     
    So I think the blogger here is ranting against the popular press (and scientists who talk to the popular press) and how it reports on findings too early and too flippantly. But he also seems to be dismissing all empirical research and, in particular, dismissing all research excepts physics. 
     
    Some of the studies described by Lehrer are rigourous meta-analyses, synthesized across multiple studies. They are not "n = 4 studies" (to use another quote from the blog post).  In this post I do not see a substantive criticism of the studies that Lehrer discusses as a basis for his views.
     
    As somone interested in the exchange of knowledge between science and practice, I come away thinking that the blog author believes there is no truth and no value in scientists trying to explain their findings to a broader public. This seems to me to be a nihilistic view of all social science, rather than a targeted criticism of the Decline Effect.
     
    Maybe I missed something?
     
    Ken
     
    ~~~~~~~~~~~~~
    Kenneth G. Brown, Ph.D., SPHR
    Associate Professor of Management & Organizations and Tippie Research Fellow
    Henry B. Tippie College of Business
    The University of Iowa
    Iowa City, IA 52242
    Ph: 319.335.3812  Fx: 319.335.1957
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Fred Nickols
    Sent: Friday, October 14, 2011 8:13 AM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    In relation to the article Gary posted, here's one that takes a critical view of the claims made in it.
     
     
     
    Regards,
     
    Fred Nickols
    Managing Partner
    Distance Consulting LLC
    Home to "Solution Engineering"
    1558 Coshocton Ave – Suite 303
    Mount Vernon, OH 43050
     
    "We Engineer Solutions to Performance Problems"
     
     
     
     
    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.PACE.EDU] On Behalf Of Gary
    Sent: Thursday, October 13, 2011 6:37 PM
    To: MG-ED-DV@AOMLISTS.PACE.EDU
    Subject: Re: AOM 2012: Planning for a PDW/paper stream concerning the validity of using university students as a generalisable sample
     
    Colleagues,
     
    You have probably heard about the Decline Effect.
    I seems relevant to Romie's request.
     
    The Decline Effect, New Yorker Magazine.
     
    Disturbing or intriguing.
     
    Best,
     
    Gary Lundquist