|Teaching and Learning Forum 2013 [ Refereed papers ]|
Catherine Moore and Susan Teather
Edith Cowan University
Email: firstname.lastname@example.org, email@example.com
There is a growing interest in tertiary education in Australia in developing the capacity of learners to evaluate and improve both their own work and that of others (Boud & Falchikov, 2006; Oliver, 2011). In order to successfully direct their own learning beyond university (and engage in lifelong learning), students need to be able to evaluate their performance in relation to a standard, identify gaps and determine how to bridge them in order to achieve the desired standard. One strategy that can be employed to achieve this is that of engaging students actively in the feedback process, so that feedback becomes an integral part of learning. In this paper the authors define feedback as a loop, meaning that feedback can only be said to have occurred when there is some identifiable influence on the recipient of the feedback (Boud & Molloy, 2012). When feedback results in improved performance we can say that learning has occurred. This paper describes how peer feedback was embedded in the teaching, learning and assessment cycle of a third year social studies unit. Pre and post surveys of students reveal their response to the requirement that they actively engage in evaluation and feedback. This paper explores how engaging students in peer review as part of a major assessment task affected their learning, their capacity to direct their learning, and their self-efficacy in relation to academic and real-world tasks.
Feedback can motivate and improve learning so it is essential for students to be provided with effective and appropriate feedback (Pearce, Mulder & Baik, 2009). Feedback that focuses on "growth rather than grading" (Sadler, 1983, p. 60) is far more likely to advance student learning than feedback that does not make sense to students (Gibbs & Simpson, 2004).
Students are more motivated to engage with and use feedback when the immediate utility of that feedback is clear. They see the purpose of feedback as something that helps them do better in the next task or, when feedback is given on drafts, something that can immediately be used to improve the final product (Price, Handley, Millar & O'Donovan, 2010). The difficulty students have in interpreting feedback was also highlighted by Price et al. (2010). They report that written feedback without dialogue is often viewed by students as frustrating and can lead to disengagement. They further note student perceptions that dialogue and discussing examples of work assists interpretation of feedback and increases the utility of feedback.
Black and Wiliam (1998) refer to the substantial evidence from research that demonstrates classroom assessment will most effectively enhance learning when learners:
This paper outlines a feedback process involving several related activities which facilitated student understanding of the criteria by which their work would be judged, developed their capacity to identify both current and desired levels of achievement, generated information about how to close the gap, and gave them the opportunity to use that information to actually close the gap. We believe this to be a sustainable feedback process (Hounsell, 2007) as it contains the four characteristics of sustainable feedback identified by Carless, Salter, Yang and Lam (2011):
Students were surveyed to gauge their response to the peer review process in which they had engaged in dialogue about what constitutes quality, monitored and evaluated their own learning and that of others, and had ultimately used the feedback that they gave, and received, to enhance their achievement in a two-stage major assignment.
The intention was to facilitate a paradigm shift from student to professional social worker (which involved students becoming empowered rather than merely engaged) by developing their ability to monitor, manage and evaluate their own learning, and to reflect the collaborative ways of working students would encounter after graduation.
The authentic assessment in the unit assists students to develop a number of graduate attributes and employability skills. The development of graduate attributes such as effectively communicating with others, working in teams, generating ideas, considering cross-cultural and international perspectives, and critical reflection and appraisal skills are focused upon in this unit. The development of these attributes is closely linked to the identified essential employability skills that these final year students require as they soon embark upon their new career.
The planned learning activities in this unit were based around authentic assessment in a situated learning context, which encouraged active and deep learning (Bonwell & Eison, 1991; Lave & Wenger, 1991; Wiggins, 1990). The assessment consisted of two linked assignments which asked each individual student to construct a full family support program for a real population. Students were given agency in selecting the population for which they designed the program. In selecting learning activities relevant to the students and in their area of interest, independent learning is enhanced (Candy, 1991).
In order to ensure the learning led to higher levels of cognitive performance, teaching strategies that promote active learning were embedded extensively throughout the unit. These included common elements cited by Bonwell and Eison (1991) such as student involvement in discussing and writing rather than just listening, emphasis on skill development rather than content transmission, and frequent use of higher order thinking skills such as synthesis and evaluation rather than those of explanation and application. Preparation of students began from the commencement of the semester when students were informed that they would be required to be actively involved in conversations involving critical thinking during the semester. On campus students spent class time discussing their assignment with their peers which provided the opportunity for the students to critically consider and appraise which created deeper understanding of their work. Off campus students were encouraged to actively engage in this unit's Discussion Board so that they also had the opportunity to share with and learn from their peers. These opportunities enabled the students to be active learners as they listened and evaluated in dialogue with their peers. Conscious of the critical importance of effective formative feedback, and the value of engaging students in that process, a peer reviewing process was introduced. This was designed to offer students further opportunities to enhance the development of graduate attributes and a range of employability skills whilst they were engaging in meaningful, relevant and deep learning.
A contributing factor in the decision to use a peer review process was the unsustainable lecturer workload. The assessment for the unit consisted of two linked assignments. In previous years the method of providing detailed, constructive and individual feedback to students on the first assignment, so that they could revise and improve their work for the second assignment, created a significant workload for the lecturer. Whilst the provision of extensive feedback was considered pivotal to the students' learning and improvement in skills, and the importance of the feedback was acknowledged by the students, it would have been challenging for the lecturer to sustain this in future years, particularly in the face of growing student numbers.
Furthermore, as responsibility for directing the learning required to close the gap between current and desired performance fell to the lecturer rather than the students, the lecturer noted that students tended to adopt a fairly passive approach to the revision of their tasks. Although marking guides were supplied with tasks, the interpretation of these guides fell to the lecturer while students passively awaited feedback. Therefore, students never fully developed an understanding of what the various levels of performance actually looked like. This realisation contributed to exploring alternative ways of providing quality feedback, whilst enhancing student learning, involving them more as independent learners and developing the very important employability skills and graduate attributes.
The marking guides for the assessment were adequate for the lecturer's marking, but they were not as effective a learning tool for students as they could be. In order for students to be more independent in their learning and to be more in control of their learning the marking guides needed to be improved so that students could accurately interpret them and develop a clear understanding of the learning goals and standards of performance required for each task. This was particularly necessary for the peer reviewing process. For a student to review a piece of work they had to have a clear understanding of what was required. Whereas in the past the lecturer had interpreted the criteria for the students in the process of providing written feedback, this was not appropriate for peer reviewing where student would need to provide the feedback. Consequently, a more comprehensive marking guide was developed that was easier for students to interpret.
With the motivation to implement a change and the development of a useful marking guide the path was set for the introduction of peer reviewing. Prior to this, planning and contemplation of potential challenges was essential to success. Implementing changes produced a number of challenges for the lecturer, such as how to overcome student resistance to openly sharing their work with their peers. This resistance was not surprising as Drury, Kay and Losberg (2003) found that, even after explicit teaching of group work skills and structured discussion and reflection on group processes and dynamics, a significant minority of first year students report negative experiences of group work. Perceived demands that students compete with each other for marks can also create resistance to sharing their work with their peers. Discussions in class revealed that students feared that their work would not be good enough and that they would be criticised. Yet for graduates of this course, working collaboratively with peers in the workplace is essential. Therefore, the lecturer needed to be aware of the challenges that students may encounter and their perceptions and then had to ensure that she introduced strategies to address the hesitation and resistance of the students. Clear and open communication of the process and the benefits for the students, as well as ongoing support and encouragement by the lecturer, were utilised to ensure students understood the value of being involved in the peer reviewing process.
Engaging students in reviewing their peers' work required considerable contemplation, planning and preparation, particularly around the actual feedback. The development of support materials started several months before the semester began. The lecturer developed instructions detailing the process and how to provide the feedback so that all students had written guidelines of what was expected of them in the peer review process. On-campus students were given both verbal and written instructions of the process and time was spent, from the beginning of semester, explaining the process and the expectations of the students. Off-campus students were encouraged through Discussion Board and email contact to discuss their concerns and to ask questions in order to gain a clear understanding of the processes.
Time and timing were critical to success. A timeline (see Appendix 1) was set for the semester to ensure that students had sufficient time to learn about and to reflect on the peer reviewing process. The process was not rushed as it was planned prior to the commencement of the semester. This long lead in allowed the lecturer time to contemplate and to consider adjustment and refinements to the process and the supporting documents, such as the guidelines and the marking guides. Students were introduced to peer reviewing as soon as they engaged with the unit. Detailed written information was provided and both on and off campus students were involved in discussions about peer reviewing from the start of the semester. Peer reviewing was planned for and it was explained to the students extensively.
All aspects of this unit relating to peer reviewing were planned for prior to the commencement of the semester. This included preparing students for their involvement in the surveys. On campus students were informed both verbally and in written format that they would be asked to be involved in completing two surveys. Similarly, off campus students were also informed via Discussion Board and emails. At the point of administering both the pre and post surveys, on campus students were provided with paper versions and off campus students were emailed the survey.
The pre-survey elicited responses in relation to students' prior experiences of peer review, their response to the preparatory peer review activity conducted using work samples, and their feelings about the review of their peers' work that they would be undertaking a few weeks later. In order to gather this information prior to peer reviewing, this survey was distributed to the students in week four. It was completed immediately after students had undertaken an activity involving peer review of work samples. To cater to the different modes of study, on campus students completed the survey in class whereas off campus students were provided with the survey form to complete electronically. The pre-survey gathered both qualitative and quantitative data and included six questions, with four containing a rating scale as well as the opportunity to provide comments and the remaining two questions requiring comments only.
Immediately prior to the pre-survey being distributed, students were involved in reviewing a sample assignment. The sample assignment was provided to on campus students in class, whereas off campus students were able to access the assignment via BlackBoard. The on campus students were given the peer reviewing key, based upon the marking key, and guidelines providing detailed instructions on the peer reviewing process. Students were allocated time to read the guidelines and ask any questions in order to clarify what was required and address any concerns they might have. After they completed this they started to review the sample assignment. This process took 30-40 minutes and after all students had completed this task individually students were gathered into groups of three to four and began discussing the sample assignment. In these discussions students were required to justify their comments on the peer reviewing key. They were also encouraged to question each other and to seek clarification from their peer as to their comments.
Similarly, off campus students engaged in the same process but due to their mode of study students engaged in discussions via Discussion Board. The lecturer encouraged involvement and discussion by monitoring Discussion Board and providing comments were necessary. Off campus students were not limited in time to complete this learning task.
The post-survey canvassed students' views on the usefulness of the peer review exercise and their feelings about engaging in such activities in the future. This survey was administered at the end of the semester in week 12, after students had provided feedback to a peer and had responded to that feedback. Similar to the pre-survey, students undertaking their studies on campus completed the post-survey in class and off campus students were contacted electronically. The post-survey also utilised the collection of qualitative and quantitative data. This survey contained 14 questions, some of which had rating scales and opportunities for comments while others only required comments.
Both on and off campus students completed the first survey. There were 15 on campus and three off campus surveys completed. The first survey canvassed students' prior experiences of peer review and elicited responses regarding the perceived usefulness of various aspects of the practice peer review process (which used work samples). It also asked students how they felt about reviewing another student's work and how they felt about having another student review their work.
Again the second survey was distributed to both on and off campus students. This time 12 on campus students completed the survey. Unfortunately, to date no off campus students have responded to the second survey. Students were asked about their experience of peer reviewing and what they learnt. In particular, they were asked about the experience and perceived value of providing and receiving feedback. The data gathered from these pre and post surveys were collected, collated and analysed.
Students liked the opportunity to
Evaluating work samples
Question 2 asked students how useful they found the experience of evaluating the work samples. All students appeared to find the experience useful, with five rating it as fairly useful, seven as very useful and 3 as incredibly useful.
Students who found it incredibly useful liked
Students also found that evaluating work samples
Students commented that the experience:
Students' lack of confidence in their ability to identify areas for improvement was accompanied by comments that
Question 5 asked: How do you feel about reviewing another student's work?
Most students expressed some apprehension, with responses like:
Learning from the experience
Determining the usefulness and the degree of learning students gained from the process was explored in Questions 2-7. Students were asked what aspect of the peer reviewing they learnt the most from. A total of 83.3% of students indicated that the process of both giving and receiving feedback provided the most learning. Comments from students were insightful and positive. For example one student commented that, "My way of giving feedback was a learning experience as I hadn't formally done anything before. I also learnt how to receive feedback without being closed minded."
Further comments from students were that they learnt from the process, the areas that they needed to focus upon to improve their assignments. "I learnt from the strengths in my peers' assignment, and realised how important it was that all parts of the program were linked". Also one student noted that peer reviewing, "Was useful to see where improvements can be made on my own assignment and was good to learn about another issue/program."
A number of students (75%) commented that they appreciated the suggestions they received from their peers. Students were then asked if they would modify their assignment as a result of the peer review activity and there was a response of 91.6% saying that they would do this.
Giving and receiving feedback
A range of questions (Questions 8-12) were presented to the students about giving and receiving feedback. Students were asked about the quality of the feedback that they received from their peer. A total of 83.2% of the students indicated that the quality was excellent (41.6%) or good (41.6%). The remaining 16.8% indicated that the feedback was fair. Feedback provided by the students was also surveyed and 91.6% indicated that they thought the feedback they had provided was excellent (25%) and good (66.6%). One student (8.4%) indicated that she thought she had provided poor feedback.
Students were also asked to indicate if they thought they were prepared for the task. Almost all students (91.6%) indicated that they considered that they were. Only one student said that she wasn't prepared for the task. It is worth noting that this student was the same student who indicated that her feedback to her peer was poor and who had a poor attendance record.
How students felt about reviewing and about having another student review their work was explored. Some comments from students indicated that they were apprehensive about the task prior to engaging in the process, but they did realise that the task was useful:
Peer reviewing provides students with the opportunity to encounter greater diversity of perspectives. As indicated by Pearce, Mulder and Baik (2009, p. 3) this is an important benefit of peer reviewing. Through engaging in this experience students received feedback from two people instead of the just the lecturer. Whilst the task of marking and assigning a grade to the assignment remained with the lecturer, the two perspectives offer diversity which is very useful for the students, and reflects the way their work would most likely be reviewed as future professional social workers.
An additional benefit relates to providing feedback. As students engaged in this process they were required to analyse, review, clarify and then verbally provide respectful and meaningful feedback to their peer. The benefits of involvement in the process of reviewing and providing feedback have been acknowledged by many (see for example, Lundstrom & Baker, 2009; van den Berg, Admiraal & Pilot, 2006). It is not just receiving feedback that is beneficial; it is the giving of feedback that provided clarity for some students. Students reported that they benefitted from engaging in this process as it enabled them to interact with their peers, to discuss their reviewing and to provide a different perspective. This different perspective further assisted some students view their assignment differently. Comments indicated that students appreciated the feedback they received and were going to utilise this information to adapt and improve their second assignment. Providing and receiving feedback had been repositioned as an effective learning tool.
Additional benefits to emerge from engaging in peer reviewing were the development or expansion of a range of generic skills. Pearce, Mulder and Baik (2009) suggest that engaging in peer reviewing provides opportunities to develop diverse skills such as problem solving and reflection. It is also suggested that peer reviewing increases a sense of responsibility, promotes independent learning, reduces dependence on the lecture and, very importantly, prepares students for the professional workplace. Students' engagement in this process provided them with opportunities to develop the diverse skills that Pearce, Mulder and Baik (2009) identified.
With a clear focus upon graduate attributes, this experience provided a comprehensive opportunity for students to enhance their employability skills. Engagement in a collaborative learning environment that provided authenticity enabled students to learn skills to assist them as they enter their profession. Reviewing of peers' work within the workplace is common within the community work sector. Having the opportunity to encounter this and to learn about the process of reviewing is beneficial to graduating students. Throughout the semester students work within groups on a range of activities where it is necessary to communicate respectfully and clearly both verbally and in written format. They are required to critically consider ideas and concepts, to question and to generate ideas. As students within this unit focus upon a population of their choice a number of students chose populations from different international and cultural perspectives. Again, exposure to different cultural perspectives is important for graduates who will potentially be working with diverse communities. Peer reviewing provided the ideal opportunity to develop and/or expand a diversity of skills and enhance a range of attributes essential for graduates.
The benefit of engaging students in peer reviewing extends beyond the students. Lecturers require considerable time to provide appropriate and meaningful feedback that is essential for students learning (Boud & Molloy, 2012). Through the use of peer reviewing and the comprehensive marking guide created for this task, the allocation of marking time required by the lecturer was reduced. Whilst this was not the primary aim of the introduction of peer reviewing, it was a positive consequence. This outcome, together with the outcomes for the students, further emphasises the overall benefits of peer reviewing. And as Pearce, Mulder and Baik (2009) have suggested, peer reviewing is multidimensional, providing diversity in students learning and various benefits.
Some limitations of this research relate to the gathering of the data. Whilst the survey was distributed to both on and off campus students, the return of the survey from off campus students was far less than on campus students. Consequently there is an impact on the data collected. In the future further consideration will be given to engaging the off campus students in the data collection process. It would beneficial to gain more information from the students undertaking this mode of study as the experience of these students may be different to the on campus students.
The survey data collected to date draws attention to the need for careful preparation of students for both giving and receiving feedback. While the importance of respectful and constructive feedback was emphasised before and during the work sample activity, there was no real focus on how students should receive feedback. This is an area which the researchers would like to refine for future activities.
This process of peer reviewing gave students the opportunity to engage in deep learning within an environment that emulated the workplace. This is particularly important to these final year students who are about to enter their chosen profession. They have, within this unit, had the opportunity to develop a range of attributes and skills that are essential for this next step of their learning journey.
Our experience was that student contact time spent in discussion and reflection about all aspects of the process was extremely valuable. Students' perceptions of the value of the peer review exercise were shaped during these discussions and their view of themselves as professional social workers was enhanced. Students were able to see clear connections between their ways of working and ways of being in the classroom and their future professional social work roles.
Engaging students in further opportunities to peer review would expand the benefits to students' learning and the development of a range of skills important for these potential graduates. There is evidence in the literature and also from the student surveys conducted in this research reinforcing this. Therefore, by expanding the use of peer reviewing to more than one isolated unit within a degree, learning and skills development would be expanded and reinforced. As student became more familiar with peer reviewing the issues initially confronted would be reduced as students would be experienced with the process and therefore less intimated.
Bonwell, C. C. & Eison, J. A. (1991). Active learning; Creating excitement in the classroom. ERIC Digest. http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED340272
Boud, D. J. (2012). New conceptions of feedback and how they might be put into practice. Keynote address presented at eAssessment Scotland 2012 Conference, University of Dundee. [verified 23 Jan 2013; PowerPoint slides] http://www.srhe.ac.uk/downloads/events/69_Boud.pdf
Boud, D. J. & Molloy, E. (2012). Rethinking models of feedback for learning: The challenge of design. Assessment and Evaluation in Higher Education. http://dx.doi.org/10.1080/02602938.2012.691462
Boud, D. J. & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education, 31(4), 399-413. http://dx.doi.org/10.1080/02602930600679050
Candy, P. C. (1991). Self-direction for lifelong learning. Jossey-Bass Higher and Adult Education Series San Francisco, California.
Carless, D., Salter, D., Yang, M. & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407. http://dx.doi.org/10.1080/03075071003642449
Drury, H., Kay, J. & Losberg, W. (2003). Student satisfaction with groupwork in undergraduate computer science. Australasian Computing Education Conference. http://crpit.com/confpapers/CRPITV20Drury.pdf
James, R., Krause, K.-L. & Jennings, C. (2010). The first year experience in Australian universities: Findings from 1994 to 2009. http://www.cshe.unimelb.edu.au/research/experience/docs/FYE_Report_1994_to_2009.pdf
Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.
Li, L., Liu, X. & Steckelberg, A. L. (2009). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536. http://dx.doi.org/10.1111/j.1467-8535.2009.00968.x
Lundstrom, K. & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer's own writing. Journal of Second language Writing, 18(1), 1-43. http://dx.doi.org/10.1016/j.jslw.2008.06.002
Nicol, D. (2009). Transforming assessment and feedback: Enhancing integration and empowerment in the first year. In Quality Enhancement Themes: The First Year Experience. The Quality Assurance Agency for Higher Education, Mansfield, UK. http://www.reap.ac.uk/reap/public/papers//QAA_DN_Paper_final_08.pdf
Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. http://dx.doi.org/10.1080/03075070600572090
Oliver, B. (2011). Assuring graduate outcomes. Australian Learning and Teaching Council. http://www.olt.gov.au/system/files/resources/Assuring%20graduate%20outcomes.
Pearce, J., Mulder, R. & Baik, C. (2009). Involving students in peer review: Case studies and practical strategies for university teaching. University of Melbourne: Victoria. http://www.cshe.unimelb.edu.au/resources_teach/teaching_in_practice/docs/Student_Peer_Review.pdf
Price, M., Handley, K., Millar, J. & O'Donovan, B. (2010). Feedback: All that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3), 277-289. http://dx.doi.org/10.1080/02602930903541007
Rust, C., O'Donovan, B. & Price, M. (2006). A social constructivist assessment process model: How the research literature shows us this could be best practice. Assessment and Evaluation in Higher Education, 30(3), 231-240. http://dx.doi.org/10.1080/02602930500063819
Van den Berg, B. A. M., Admiraal, W. F. & Pilot, A. (2006). Design principles and outcomes of peer assessment in higher education. Studies in Higher Education, 31(3), 341-356. http://dx.doi.org/10.1080/03075070600680836
Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2). http://pareonline.net/getvn.asp?v=2&n=2
|Week 1||Details of peer reviewing provided to students.|
|Week 2||Students engaged to look at the guidelines to be used in the peer reviewing process.|
|Week 3||Lecturer considers feedback from previous week and if there is any, adjustments are made to peer reviewing document.|
|Week 4||Students provided with sample assignments and Peer Reviewing Key. On campus students undertake a review of these assignments in class. Off campus students engage in the same process through using Blackboard.|
Pre-survey distributed to students.
|Week 6||Students engage in the peer reviewing process in class and online.|
|Week 8||Students are provided with a copy of the review of their work when marked assignments returned.|
|Week 12||Students submit their final assessment in this week. Students are required to consider the feedback provided by their peer. Whilst they do not have to use this feedback the student must consider it and reflect on the feedback, explaining why used or why they didn't use the feedback.|
Post-survey distributed to students.
|Please cite as: Moore, C. & Teather, S. (2013). Engaging students in peer review: Feedback as learning. In Design, develop, evaluate: The core of the learning environment. Proceedings of the 22nd Annual Teaching Learning Forum, 7-8 February 2013. Perth: Murdoch University. http://otl.curtin.edu.au/professional_development/conferences/tlf/tlf2013/refereed/moore.html|
Copyright 2013 Catherine Moore and Susan Teather. The authors assign to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.