Susan Teather and Catherine Moore
Edith Cowan University
s.teather@ecu.edu.au, c.moore@ecu.edu.au
Professional practice
[ Refereed papers ]
| |
| Embedding innovative assessment practices: Developing students' critical appraisal skills
Susan Teather and Catherine Moore |
Category: Refereed Professional practice [ Refereed papers ] |
There is a growing imperative in tertiary education (nationally and internationally) to produce graduates who can work collaboratively to assess and improve their own work and that of others (Bologna Process, 2010: Boud & Falchikov, 2006; Oliver, 2011). At ECU our new undergraduate curriculum framework titled Curriculum 2012: Enabling the learning journey promotes assessment for learning. This develops both the capacity to learn and the ability to direct learning. In order to successfully direct their own learning beyond university, students need to be able to identify the standard of performance to which they should aspire as a result of that learning, accurately locate where they are in relation to the standard, and then develop pathways to bridge the gap. In other words they need to engage in formative assessment.This paper reveals how one lecturer introduced innovative practices in teaching and assessment in order to enhance her students' ability to direct their own learning, to increase the value students place on their feedback, to ensure their active engagement with feedback, and ultimately to develop students' ability to calibrate their own judgement about their learning. The innovation was designed to increase student understanding of what constitutes academic rigour within the discipline as well as the standards required for success in real world endeavours.
Keywords: critical appraisal, assessment, feedback, formative, academic rigour, lifelong learning
This unit seeks to develop the graduate attributes of effectively communicating with others, working in teams, generating ideas, considering cross-cultural and international perspectives; and critical reflection and appraisal skills. The development of these attributes is essential to these final year students who will in the near future graduate and embark upon their new career.
The planned learning activities in this unit are based around authentic assessment tasks in a situated learning context, thus encouraging active and deep learning (Wiggins, 1990; Lave & Wenger, 1991; Bonwell & Eison, 1991). The assessment tasks require that each student identifies a population for which they will design a programme, giving students agency in selecting learning activities relevant to them and their area of interest, and enhancing independent learning (Candy, 1991). After this initial identification of a target population, the students embark upon designing a family support programme for this group. This task is complex, involving many aspects, but each is introduced through the content of weekly lectures, tutorials and comprehensive modules to ensure that students develop a full understanding of all the requirements of designing a family support programme.
Students commence this unit by revisiting social science theories and reviewing literature focused upon the needs of families and the variables that will support and address these needs. They develop their knowledge and understanding of different populations and the importance of creating programmes designed to support the individual needs of families within various populations. Students learn to combine practical knowledge with the theories underpinning family support that are essential to embrace when working effectively and respectfully with families.
Bronfenbrenner's ecological theory (Bronfenbrenner, 1979) suggests that to fully understand a developing child, it is essential to consider all the factors that impact upon that child, such as their family, friends, neighbours, school, community environments, services whilst also extending to include societal attitudes and government policies. By extension then those factors, such as cultural background, socioeconomic status, religious beliefs, education and work status which affect the family, also need to be considered as these influence the family's approach to raising children. Understanding the links between these factors, and the strength of relationships or connections between them is also critical to the design of effective family support programmes. There is no such thing as a decontexualised child and therefore, the child must be considered in the context of all the environments they are involved in (Bronfenbrenner, 1979). The people within these environments that directly link to the child are influenced by other environments and other people, whilst also influencing these environments and people. Therefore, it is essential that students demonstrate their understanding of all these factors as they develop a family support programme.
Bronfenbrenner's ecological theory also influenced the introduction of the innovative assessment practice in this unit. Students bring to their learning environment their individual characteristics such as readiness to learn, culture and self-efficacy, which impact upon their development as a person (Yih Chyn Kek, Darmawan & Chen, 2007). In addition, students' individual characteristics impact upon the connections they have between various systems they are involved in. The application of the theory continues as students explore the impact of culture, societal attitudes and government policy upon the people they are designing the family support programme for. Awareness of this enhances students' understanding of the application of the ecological theory and further develops the skills that will assist them as graduates. Overall, students are immersed in a learning environment that is authentic and clearly links theory to practice.
Consistent with Lave and Wenger's (1991) description of situated learning as a model of learning within a community of practice that is based on authentic problem-solving requiring interaction with others and with environments, the assessment for this unit asks students to design a family support programme for a real population of their choice. This opportunity to choose means, for example, that:
With high numbers of students the workload for the lecturer of providing extensive feedback for each student was extremely high and not sustainable. Responsibility for directing the learning required to close the gap between current and desired performance also fell to the lecturer rather than the students. Although marking guides were supplied with tasks, the interpretation of these guides was done by the lecturer while students passively awaited feedback. Therefore, students never really developed an understanding of what the various levels of performance actually looked like.
The lecturer also wished to assist her final year students to realise their potential as future leaders. This motivated the lecturer to create a learning experience that would develop their ability to lead others in their practice. To accomplish this, the skills of critical reflection and appraisal are essential and this unit provided an ideal opportunity for students to attend to these skills.
With a desire to address the challenges of high teacher workload in giving high quality feedback, lack of student engagement with feedback, and a desire to promote lifelong learning, collaborative and leadership skills in graduates, it became clear that changes needed to be made to the assessment strategy. To begin this change process the lecturer contacted an academic advisor in the Centre for Learning and Development at Edith Cowan University and a productive collaborative relationship began. An initial analysis of the assessments for this unit was followed up with further meetings. Further reflection by both partners, and a willingness to achieve a desired outcome, led to more substantive changes being embarked upon which centered on the implementation of peer review processes.
The retention of the assessments ensured that students continued to draw upon Bronfenbrenner's ecological theory as they applied the theory through the designing of a family support programme and as they engaged in contextualised learning. As proposed by this theory, the interactions and relationships that an individual has with their environments are influential to outcomes (Bronfenbrenner, 1979). Therefore, it was important to ensure that students continued to apply this theory in their assignments whilst being engaging in learning that embraced interactions with peers. This approach reinforced the underpinnings of this theory that were essential to the assessment whilst also being essential to the students learning environment and therefore their outcomes.
In order to ensure deep learning leading to higher levels of cognitive performance, teaching strategies that promote active learning were embedded even more extensively throughout the unit. These included common elements cited by Bonwell and Eison (1991) such as student involvement in discussing and writing rather than just listening, emphasis on skill development rather than content transmission, and frequent use of higher order thinking skills such as synthesis and evaluation rather than those of explanation and application. To do this students draw upon a range of skills that are essential employability skills (DEST, 2002). Students must work with their peers both in class and on-line. This creates authenticity as within any workplace communication happens within groups, some are face-to-face and others involved virtual communication. Students discuss their ideas and concepts with peers, questioning and challenging each other, therefore enhancing their critical reflection and appraisal skills. All of which are essential attributes for students to possess and to utilise in their career and future workplaces.
A further challenge was ensuring that students were actively participating in their learning. In some incidences students become dependent upon their lecturer for feedback, not seeking the consult with their peers and to actively engage in constructive and reflective processes. Students in this unit were being asked to take on a leadership role and to provide constructive and respectful comments to their peer. They were being asked to actively participate in this process that in the past had been reserved by the lecturer. This active participation is another important factor for students to embrace, as within workplaces graduates will be expected to take on leadership's roles and to actively engage and work with their peers. Students could have considered that providing feedback to their peers was not part of their role as a students, but clear communication of the process and clear explanation of the concept of peer reviewing ensured that students were well informed and therefore actively engaged (Cathy do you have a reference here about doing this?). This involvement from the beginning of semester ensured that this potential challenge was addressed and did not emerge as an issue later on.
Engaging students in reviewing their peers work required considerable contemplation, particularly around the actual feedback. The lecturer provided instructions detailing the process and how to provide the feedback. On-campus students were given both verbal and written instructions of the process and time was spent, from the beginning of semester, explaining the process and the expectations of the students. Off-campus students were encouraged through Discussion Board and email contact to gain a clear understanding of the processes. All students were provided with guidelines of what was expected of them in the peer review process. These guidelines indicated that students must "provide considered, professional and respectful comments to your peer". Enforcing this was considered a potential challenge, yet again this did not emerge. The lecturer considers that the level of communication and the clear student expectations were the key contributing factor.
Anticipation of potential issues was important to the overall process. Through this, strategies could be set in place at the very beginning of semester. This was effective as issues did not emerge.
The innovation incorporated factors such as willingness, openness, preparedness, timing and open and clear communication. These were considered essential factors to the success of the proposed innovation and therefore the possession of these by the lecturer was critical. The first factor was the lecturer's degree of willingness to facilitate the learning process required for peer feedback. The lectures openness to change was a vital precondition to embedding the innovation successfully and this was demonstrated by her willingness to work collaboratively and consider 'possibilities' suggested by her colleague, the academic advisor.
In addition to willingness, preparedness and the allocation of sufficient time to reflect upon the innovation, was another essential factor to successful implementation. The lecturer took time to contemplate the implementation, prepare herself and then prepare the students. A timeline was established with sufficient time allocated to reflect and make necessary adjustments. Students had sufficient time to prepare themselves for their involvement in the innovation. Clear communication of the processes and the requirements were provided at the beginning of semester, and communication continued until after the peer reviewing session in week seven.
Additional to these factors, the ability of the lecturer to implement the peer reviewing process was critical to success. The collaborative working relationship again contributed to achieving this as the lecturer engaged in conversations with her academic advisor. She was introduced to literature to help her increase her understanding of peer reviewing and engaged in constructive conversations. Success was also impacted upon the lecturer's ability to communicate the process and to engage students in this aspect of their learning. Experience of working with students and having developed relationships with students contributed to the lecturers' ability to facilitate this process successfully. To enhance the innovation and to ensure rigour, Lave and Wenger's (1991) idea of situated learning was encapsulated into the two CHN3206/4115 assessments. There was a natural fit between situated learning and these assessments as the assignments emphasised authentic problem-solving with others and within environments. The alignment between situated learning approaches and practices within this Social Science unit is illustrated in Table 1 (below).
| Situated learning | CHN3206/4115 |
| Authentic contexts reflect the way knowledge will be used in real life | Students design a family support programme to meet a real need in a real situation |
| Authentic activities reflect future professional activities | Students obtain feedback from others in designing the programme |
| Access to experts | Research, lecturer input, community input |
| Modelling of authentic processes | Scaffolded assignments mimic development stages of real programmes |
| Multiple roles and perspectives in the collaborative construction of knowledge | Students provide feedback to peers and gain the perspective of peers on their work |
| Promote articulation to enable tacit knowledge to be made explicit | Students discuss required standards of performance and what they might look like |
| Reflection to enable abstractions to be formed | Written feedback and reflection on feedback in order to respond |
Time and timing were critical to success. Therefore, a timeline (see Table 2 below) was set for the semester to ensure that students had sufficient time to learn about and to reflect on the peer reviewing process. The lecturer explored and read about peer reviewing to increase her knowledge of the process and the benefits of engaging in this within the learning environment. Furthermore, the lecturer continued to consult with the academic advisor to ensure the quality of the peer reviewing process being introduced.
| Week 1 | Details of peer reviewing provided to students. |
| Week 2 | Students engaged to look at the guidelines to be used in the peer reviewing process. |
| Week 3 | Lecturer considers feedback from previous week and adjustments are made to peer reviewing document. |
| Week 4 | Students provided with sample assignments and Peer Reviewing Key. On-campus students undertake a review of these assignments in class. Off-campus students engage in the same process through using Blackboard. |
| Week 7 | Students engage in the peer reviewing process in class and online |
| Week 9 (approximately) | Students are provided with a copy of the review of their work when marked assignments returned. |
| Week 13 | Students submit their final assessment in this week. Students are required to consider the feedback provided by their peer. Whilst they do not have to use this feedback the student must consider it and reflect on the feedback, explaining why used or why they didn't use the feedback. |
To further assist students' understanding and the expectations, guidelines of the peer reviewing process were developed. Time was allowed to develop and then reflect upon these guidelines but to ensure their accuracy and the effectiveness the lecturer recruited five students (both on and off-campus as instructions differed slightly depending upon students mode of enrolment) to review them. Feedback from the students identified that some points requiring further clarification. For example, the time frame for the reviewing was not considered clear and the guidelines for the off-campus students required further clarification around the actual process of reviewing. These comments resulted in a revision of the guidelines. These were again shown to two on-campus students to clarify that the changes made were sufficient.
Following this initial stage, in week four the students were introduced to peer reviewing in class and on-line. On-campus students were provided with sample assignments from previous years and the peer reviewing key. To facilitate this process the lecturer explained the process and encouraged the students to first consider the assignments on their own and then to discuss in their groups. Similarly, off-campus students were provided with sample assignments and peer review key and encouraged to go through the same process but to use Discussion Board as the means of discussion with peers. Group discussions enabled students to consider differences and similarities between comments and in relation to their assignment, consider and gaps and what they can do to rectify these prior to the submission of the assessment.
A further aim of the week four reviewing was to gather feedback on the peer review key as it was important that the document was useful and provided a high degree of clarity. Both off and on-campus students were asked to provide comments and they indicated that the document did not require any further changes.
With this task completed the next task was to undertake the actual peer review session. This session will be conducted in week seven and differed slightly to suit both on and off-campus students. On-campus students brought a copy of their assignment to class for reviewing by their peer. Students were allocated another student's assignment to review. They were given a peer reviewing key and were allocated 40 minutes in which to review the document. Once completed, students paired with the students whose assignment they had reviewed and provided verbal feedback in a respectful, reflective and professional manner. To ensure that all students had the opportunity to give and to receive feedback this was undertaken twice.
Off-campus students undertook this task similar to on-campus students. The lecturer allocated students with a peer's assignment to review. The students conducted this review and wrote comments to their peer in the peer reviewing key document. The reviewed assignment was sent back to the lecturer who then forwarded the document to the appropriate student. As students could not engage in face-to-face conversations they provided considerable comments within each criteria and a summary of their comments at the end of the peer reviewing key document.
To finalise this aspect of the assessment, students were required to reflect on the feedback receive from their peer and to provide this reflection in their second assignment. This second assignment builds on the first one, therefore having this opportunity to receive additional feedback and respond to it, enabled students to improve their performance. Students did not have to agree with their peers comments or to even implement their suggestions, but it was essential that they show that they had considered the comments and reflected upon these. This again provides an authentic learning environment for the students as within workplaces graduates are required to consider the comments of their work colleagues when working collaboratively. At times comments are implemented and on other occasions comments are discarded. The key to this learning was around 'hearing' considered comments and determining the appropriateness of this information.
The critical reflection [peer review] I completed on a peers assignment was a great exercise for it helped me expand my abilities to constructively critic, see just how very different assignments can be completed, understand just how difficult marking can be and develop self reflection skills.To this comment, this student informed the lecturer that she and the person she reviewed had continued in the peer reviewing process as they had looked at each other's second assignment for this unit and then had expanded this as they had looked at an assignment for another unit. The process of engaging students as active participants in their learning and working collaboratively with their peers was clearly evident in these students academic life.
Other comments from students similarly reflected the success of this innovation. One student commented that "Feedback on assignments was great, the best from all classes. It really helped for the second assignment." Another student stated that "The feedback provided helped me to improve on my assignment." This process of providing feedback had also provided the opportunity for the reviewing student to develop a clearer understanding of the requirements of this assessment. Students had become more engaged, developed clearer understanding of the requirements and had developed a range of skills and attributes by being involved in this innovation.
Bonwell, C.C. & Eison, J.A. (1991). Active learning; Creating excitement in the classroom. ERIC Digest.
Boud, D.J. & Falchikov, N. (2006). Aligning assessment with long-term learning. Assessment and Evaluation in Higher Education, 31(4), 399-413.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, Mass: Harvard University Press.
Candy, P.C. (1991). Self-direction for lifelong learning. Jossey-Bass Higher and Adult Education Series San Francisco, California.
DEST (2002). Employability skills for the future. A report by the Australian Chamber of Commerce and Industry and the Business Council of Australia for the Department of Education, Science and Training, Canberra.
Juwah, C., Macfarlane-Dick, D., Matthew, B., Nicol, D., Ross, D., & Smith, B. (2004). Enhancing student learning through effective formative feedback. Higher Education Academy: London, UK.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate periperal participation. Cambridge, UK: Cambridge University Press.
Li, L., Liu, X. & Steckelberg, A.L. (2009). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536.
Nicol, D.J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218.
Oliver, B. (2011). Assuring graduate outcomes. Support for the original work was provided by the Australian Learning and Teaching Council Ltd, an initiative of the Australian Government.
Rust, C., O'Donovan, B. & Price, M. (2006). A social constructivist assessment process model: How the research literature shows us this could be best practice. Assessment and Evaluation in Higher Education, 30(3), 233-241.
Van Den Berg, B.A.M., Admiraal, W.F. & Pilot, A. (2006). Design principles and outcomes of peer assessment in higher education. Studies in Higher Education, 31(3), 341-356.
Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2). http://pareonline.net/getvn.asp?v=2&n=2
Yih Chyn Kek, M., Darmawan, I.G. & Chen, Y.S. (2007). Family, learning environments, learning approaches, and student outcomes in a Malaysian private university. International Education Journal, 8(2), 318-336. http://ehlt.flinders.edu.au/education/iej/articles/v8n2/Kek/paper.pdf
Yorke, M. (1998). Assessing capability. In J. Stephenson & M. Yorke (Eds), Capability and Quality in Higher Education. London: Kogan.
| Please cite as: Teather, S. & Moore, C. (2012). Embedding innovative assessment practices: Developing students' critical appraisal skills. In Creating an inclusive learning environment: Engagement, equity, and retention. Proceedings of the 21st Annual Teaching Learning Forum, 2-3 February 2012. Perth: Murdoch University. http://otl.curtin.edu.au/tlf/tlf2012/refereed/teather.html |
Copyright 2011 Susan Teather and Catherine Moore. The authors assign to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.