Discussion: View Thread

  • 1.  Summary of Student Peer Review query

    Posted 07-15-2008 16:03

    Dear Charles, thanks for forwarding this message to the listserve...



    Hello all,

    A couple of days ago, I asked for some input on student peer review systems for MBA programs. Thanks to you all for sharing some great ideas and resources. Attached, you will see a summary of the responses that I received (see also attached document).

    All the points that were mentioned are indeed very interesting and reminded me of taking a number of things into account I haven't thought of before. But I guess that's why I posted this request in the first place. Some thoughts:

    Why do we want peer pressure to be developed?
    Daniel Evans pointed out that "Pushing people to participate, interact (pressure, as you suggest) sounds a bit anti-learning. You can take the horse to the water - but you can't make it drink.  Pressuring people to participate may result in 'forced' vs. 'learner pulled' participation - i.e. people participating just to get a good grade."

    Daniel is right, however, it is also clear that in every class some students free-ride on the cooperation of others and that even a very small minority of selfish individuals in a group of cooperative subjects can cause cooperation to break down or decrease significantly. Research has shown that if group members possess means to sanction opportunistic behavior, a credible punishment threat can be created that provides strong incentives for self-interested individuals to cooperate, thus rendering actual punishment unnecessary. As you can see from my writing: I am an economist by training and love to use words like pressure and punishment, which can be easily misunderstood. Educators certainly use less harsh words to describe the same phenomena and at one point I should learn to use the correct terms ;-)

    One of the big questions is how to make sure that students don't abuse the evaluations in order to hurt other students because they are in competition for grades. Marie-Elene Roberge solves this issue by acting as a mediator, that is, once a student receives a very negative evaluation from another student Marie-Elene "would be working really hard with them to address the problem and see how to resolve it". In the end, it is the privilege of the teacher/instructor to actually adjust the final grading.

    Competition vs. Cooperation
    Deborah Rhodes argued that receiving feedback from peers can be good but pointed also to the risk that a peer-review system might "move the program, at least for me, from one of collaboration and co-opertive/constructive learning to competition", especially if the results of peer-reviews are prominently published on degree certificates. There is no question that we want to foster cooperation, not competition among our students, even though it has also been argued that competition stimulates group members to outperform each other and hence promotes efficiency and innovation (cf. Beersma, Hollenbeck, Humphrey, Moon, Conlon, & Ilgen, 2003). However, this approach is probably more relevant in the work place than in the class room.
    Whether peer-reviews foster cooperation or competition remains, however, an open question.


    The Grading - how to make sure that students take the peer-review seriously?
    Many of you opposed the idea that the results of the peer-review should appear on a graduation certificate. Jim Clawson, for example, suggests that "Creating a separate peer-grading system to appear on a graduation certificate is a bold move.  I fear it will create animosity among your students, as in war when locals are conscripted to guard countrymen" and Deborah Rhodes argues that the performance measured by a peer-review system "is a function that changes with time and environment and attaching it to a certificate limits the growth of the individual in his/her new environment or under different circumstances". Both are absolutely right, I agree, the outcome of the evaluation process should definitely not be part of the final certificate. Rather, it could be integrated in grading of certain projects or lectures by giving 10% of weight out of the final score.

    Some other points
    Daniel Evans: "I've experimented with peer-pressure (peer-review) in a multicultural setting and find that it is quite difficult. In some cultures such a system is very well excepted - not so in others. This resulted in evaluation problems".

    Cynthia Roberts: "It may be helpful to involve the students in this process - particularly at the MBA level. I have found that most of the students can tell you what is acceptable performance and what is not based on their past experience. I routinely have my groups create their own contribution expectations and evaluation and when they are involved in the creation process, a much more comprehensive product results. Since it sounds like this assessment will be ongoing and extend beyond one course, perhaps one of the classes covering human resource management can take this on as a project."

    Jim Clawson: "Our experience is that 30 year old MBAs will vehemently resist being required to grade classmates. Vehemently.  They believe it is the faculty's responsibility to grade them.  Forcing them to do this, in our experience, creates very low level "buy-in" on my scale".

    Thanks to Jim for sharing his experience on 360* feedback (see attached document).


    Many questions remain open and I think we will conduct some research on this topic in order to find out how efficient student peer review systems indeed are. Any inputs, suggestions, hypotheses, research questions etc. are more than welcome. Once we have first results we will be happy to share them with the list.

    One final question brought up by Dave Kravitz on the GDO listserve concerning the AoM conference in Anaheim: The cost of transportation from L.A. International to the conference location seems very high. Does anyone know of a relatively inexpensive alternative?

    Many thanks and best wishes,
    Stefan



    Stefan Volk
    Marie Curie Research Fellow

    University of St.Gallen
    Research Institute for International Management
    Dufourstrasse 40a
    CH-9000 St. Gallen
    Switzerland

    Phone +41 71 224 2479
    Fax +41 71 224 2447
    http://www.fim.unisg.ch/

    Marie Curie Actions
    http://cordis.europa.eu/mariecurie-actions/


  • 2.  Summary of Student Peer Review query

    Posted 07-15-2008 17:22

    Hello Stefan

     

    I had intended to contribute to this discussion and your summary of what has been said is very interesting.

    I and colleagues at Lancaster University have for many years worked with peer assessment and have written several papers about our experiences of doing it, a couple which I would recommend based on research on programmes (not MBAs however) using peer assessment intended to support collaborative/participative learning are:

     

    McConnell, D. (2002) The Experience of Networked Collaborative Assessment. Studies in Continuing Education 24(1), 73-92

    And also

     

    Reynolds, M and Trehan, K (2000) Assessment: a critical perspective. Studies in Higher Education 25 (3) 267 – 278 and also

    Trehan K and Reynolds PM, 2002, 'Online collaborative assessment: Power relations and 'Critical learning'', in Networked Learning: Perspectives and Issues, (eds) Jones C and Steeples C, Springer, <st1:state w:st="on"><st1:place w:st="on">Berlin</st1:place></st1:state>, pp 279-292, ISBN: 1852334711

     

    We have found it to be a very positive way of engaging with assessment as a part of a participative approach to learning and one where in general students have fully entered into and found very beneficial to their own learning and understanding. I have to agree with some of the comments however that have been made and I recently gave a conference paper myself that explored the idea of the tyranny of participation when or if not done in a reflective, thoughtful and well facilitated way that focuses on the process as much as on the outcome of the assessment. (for anyone interested its available at http://www.networkedlearningconference.org.uk/abstracts/PDFs/Hodgson_640-647.pdf - it is not just about peer assessment but that underpins much of the learners experiences of participation in the programme that is discussed in the paper.)

     

    Also I agree that, from our experience, it is not something that can be assumed is easy as pointed out by Daniel Evans in a multicultural group – and requires a lot of time to do well and even then not without problems.

     

    Whilst a lot of the above relates to online work all the courses mentioned other than the McConnell one do peer assessment face to face as well as online and overall I think my view is it is something that however done requires time and facilitation to do well but is worth it in terms of the learning associated with doing it.

     

    Best wishes

     

    Vivien Hodgson

    Professor of Networked Management Learning

    Management Learning and Leadership

    <st1:place w:st="on"><st1:placename w:st="on">Lancaster</st1:placename> <st1:placetype w:st="on">University</st1:placetype></st1:place>

    <st1:city w:st="on"><st1:place w:st="on">Lancaster</st1:place></st1:city>

    LA1 4YX

    <st1:country-region w:st="on"><st1:place w:st="on">UK</st1:place></st1:country-region>

     

    e-mail v.hodgson@lancaster.ac.uk

    telephone: +44(0)1524 594020

    fax: +44(0)1524 844262 

    http://www.lums.lancs.ac.uk/pages/Departments/DML

    Networked Learning Conference updates and proceedings:

    http://www.networkedlearningconference.org.uk

     

     

     

    From: Management Education and Development Discussion [mailto:MG-ED-DV@AOMLISTS.pace.edu] On Behalf Of Stefan Volk
    Sent: 15 July 2008 21:03
    To: MG-ED-DV@AOMLISTS.pace.edu
    Subject: Summary of Student Peer Review query

     


    Dear Charles, thanks for forwarding this message to the listserve...



    Hello all,

    A couple of days ago, I asked for some input on student peer review systems for MBA programs. Thanks to you all for sharing some great ideas and resources. Attached, you will see a summary of the responses that I received (see also attached document).

    All the points that were mentioned are indeed very interesting and reminded me of taking a number of things into account I haven't thought of before. But I guess that's why I posted this request in the first place. Some thoughts:

    Why do we want peer pressure to be developed?
    Daniel Evans pointed out that "Pushing people to participate, interact (pressure, as you suggest) sounds a bit anti-learning. You can take the horse to the water - but you can't make it drink.  Pressuring people to participate may result in 'forced' vs. 'learner pulled' participation - i.e. people participating just to get a good grade."

    Daniel is right, however, it is also clear that in every class some students free-ride on the cooperation of others and that even a very small minority of selfish individuals in a group of cooperative subjects can cause cooperation to break down or decrease significantly. Research has shown that if group members possess means to sanction opportunistic behavior, a credible punishment threat can be created that provides strong incentives for self-interested individuals to cooperate, thus rendering actual punishment unnecessary. As you can see from my writing: I am an economist by training and love to use words like pressure and punishment, which can be easily misunderstood. Educators certainly use less harsh words to describe the same phenomena and at one point I should learn to use the correct terms ;-)

    One of the big questions is how to make sure that students don't abuse the evaluations in order to hurt other students because they are in competition for grades. Marie-Elene Roberge solves this issue by acting as a mediator, that is, once a student receives a very negative evaluation from another student Marie-Elene "would be working really hard with them to address the problem and see how to resolve it". In the end, it is the privilege of the teacher/instructor to actually adjust the final grading.

    Competition vs. Cooperation
    Deborah Rhodes argued that receiving feedback from peers can be good but pointed also to the risk that a peer-review system might "move the program, at least for me, from one of collaboration and co-opertive/constructive learning to competition", especially if the results of peer-reviews are prominently published on degree certificates. There is no question that we want to foster cooperation, not competition among our students, even though it has also been argued that competition stimulates group members to outperform each other and hence promotes efficiency and innovation (cf. Beersma, Hollenbeck, Humphrey, Moon, Conlon, & Ilgen, 2003). However, this approach is probably more relevant in the work place than in the class room.
    Whether peer-reviews foster cooperation or competition remains, however, an open question.


    The Grading - how to make sure that students take the peer-review seriously?
    Many of you opposed the idea that the results of the peer-review should appear on a graduation certificate. Jim Clawson, for example, suggests that "Creating a separate peer-grading system to appear on a graduation certificate is a bold move.  I fear it will create animosity among your students, as in war when locals are conscripted to guard countrymen" and Deborah Rhodes argues that the performance measured by a peer-review system "is a function that changes with time and environment and attaching it to a certificate limits the growth of the individual in his/her new environment or under different circumstances". Both are absolutely right, I agree, the outcome of the evaluation process should definitely not be part of the final certificate. Rather, it could be integrated in grading of certain projects or lectures by giving 10% of weight out of the final score.

    Some other points
    Daniel Evans: "I've experimented with peer-pressure (peer-review) in a multicultural setting and find that it is quite difficult. In some cultures such a system is very well excepted - not so in others. This resulted in evaluation problems".

    Cynthia Roberts: "It may be helpful to involve the students in this process - particularly at the MBA level. I have found that most of the students can tell you what is acceptable performance and what is not based on their past experience. I routinely have my groups create their own contribution expectations and evaluation and when they are involved in the creation process, a much more comprehensive product results. Since it sounds like this assessment will be ongoing and extend beyond one course, perhaps one of the classes covering human resource management can take this on as a project."

    Jim Clawson: "Our experience is that 30 year old MBAs will vehemently resist being required to grade classmates. Vehemently.  They believe it is the faculty's responsibility to grade them.  Forcing them to do this, in our experience, creates very low level "buy-in" on my scale".

    Thanks to Jim for sharing his experience on 360* feedback (see attached document).


    Many questions remain open and I think we will conduct some research on this topic in order to find out how efficient student peer review systems indeed are. Any inputs, suggestions, hypotheses, research questions etc. are more than welcome. Once we have first results we will be happy to share them with the list.

    One final question brought up by Dave Kravitz on the GDO listserve concerning the AoM conference in <st1:city w:st="on"><st1:place w:st="on">Anaheim</st1:place></st1:city>: The cost of transportation from L.A. International to the conference location seems very high. Does anyone know of a relatively inexpensive alternative?

    Many thanks and best wishes,
    Stefan



    Stefan Volk
    Marie Curie Research Fellow

    <st1:place w:st="on"><st1:placetype w:st="on">University</st1:placetype> of <st1:placename w:st="on">St.Gallen</st1:placename></st1:place>
    Research Institute for International Management
    Dufourstrasse 40a
    CH-9000 St. Gallen
    <st1:country-region w:st="on"><st1:place w:st="on">Switzerland</st1:place></st1:country-region>

    Phone +41 71 224 2479
    Fax +41 71 224 2447
    http://www.fim.unisg.ch/

    Marie Curie Actions
    http://cordis.europa.eu/mariecurie-actions/