Reflection: Peer marking writing portfolio drafts in Class Notebook

Category: News
Date: Tuesday 23 February 2021

Tanya Fernbank, Warren Sheard, Joanne Shiel, Eve Smith

Introduction

The following reflection examines the process, benefits to students and lessons learnt from asynchronous peer marking of student writing portfolio drafts on a compulsory 60-credit, pass for progression module for students studying Engineering. The University of Leeds Language Centre provides some of the teaching, while the rest of the teaching is provided by colleagues from the SWJTU-Leeds Joint School in the Faculty of Engineering and Physical Sciences at South West Jiaotong University. The main aim of this reflective piece is to provide some insights into successful and less successful elements of this process for students working together in the classroom in China, yet with this being facilitated remotely by University of Leeds Language Centre staff. It examines some steps which were taken to mitigate parts of the peer feedback process which did not work as initially intended and discusses how to make further improvement in these areas moving forward. This blog may be of interest to anyone currently teaching online and particularly to those teaching on Transnational Education courses where students are being taught together in their classrooms, but with some tutors teaching them remotely. In addition, it might be interesting for those teaching on other future hybrid courses which involve some remote learning. Some of the ideas for asynchronous peer marking could also be developed for use on other courses.

The context

Students on the SWJTU course are at a similar level to those on IFY (International Foundation Year) courses at Leeds. The difference is that they complete the English for Engineering module as the common first year of a 4-year degree programme, rather than the results from this course being a prerequisite for entry onto their degrees. In the first term of the academic year 2020/21, tutors based in China did more work on subject-specific content, while tutors in Leeds focused primarily on language and skills work with this, but with themes related to students’ subjects of engineering or computer science. The focus will become more focused on academic study skills in the second semester.

The peer feedback tasks were put in place to give students practice with providing peer feedback for drafts of their writing portfolio assessment tasks. This assessment involves students completing a written portfolio of five assignments. The final versions of assignments two to five make up just 5% of students’ final marks, with the first assignment given for practice. This is, therefore, a relatively low stakes assignment with an emphasis on process and development rather than final output. Students submit a draft of each piece of written work, based on a given task, to their writing portfolios on OneNote Class Notebook once a fortnight. They then give each other formative peer feedback on this work. The aim of this is to encourage learners to learn how to provide feedback for their peers and thereby develop greater criticality in reading and proofreading their own writing. In this case, students provide feedback on three main areas which students are assessed on for their writing portfolios: content, organisation and language, with three to four questions to answer for each section. The tutor then has a few days to read these and either add a few general comments on how well they have conducted this feedback as well as what they have done well and areas to improve on. Following this, students have a lesson where the students discuss the feedback in their peer groups. After this, students are given a few days to re-write their draft, which they submit to their individual writing portfolios, whilst also leaving a copy of the original draft in these folders. The tasks in the portfolio will be marked together at the end of the semester, with the aim of assessing both on the quality of the individual tasks and how the students have performed overall.

Differences between online peer marking on this/ other courses.

One of the main challenges when setting up peer marking for online courses is considering how to do this in a way which will give students enough support in how to complete it. They also need sufficient initial training and time to be able to do this well. Although many tutors already do peer marking in small groups in face-to-face classrooms, this has not always been embedded as a compulsory activity in some courses which we have taught on in the past. This was the first time that some tutors had been involved in peer marking activities where students will use feedback from these sessions to improve their writing for assessment. Likewise, none of this team had previous involvement with peer marking through Class Notebook, as this had been done in other ways on some courses that tutors had taught on since beginning online teaching in March 2020, particularly when tutors were still learning to use Class Notebook. For instance, on previous courses, some tutors had created links to shared documents on OneDrive. However, since students are doing all their other work on Class Notebook, it makes sense to make more use of the collaborative space on this page for written peer feedback too.

Rationale for the task

The rationale behind the development of these peer marking activities is twofold. Giving peer feedback in this way encourages students to review each other's work autonomously, with the additional benefit of reducing teachers’ marking workload, since there are a total of 10 tasks, with two versions of each of the five portfolio tasks, and groups are large. It is beneficial for students to practise peer marking in the same way as they would in the classroom, in order to develop the skills of noticing what has been written well and areas for improvement in terms of content, organisation and language which impede communication of meaning. This also encourages students to give feedback in English and learn how to do this constructively rather than saying everything is fine to be polite or because they cannot identify areas for improvement. Furthermore, this can enable the student receiving the feedback to understand which areas they need to work on in order to improve their own writing. Another reason for doing this was to measure or facilitate 'engagement' with the process of writing when the teachers were not physically present. One of the advantages of setting up these files in the collaborative document in Class Notebook, OneDrive or SharePoint is that as well as being able to see the work of those in their peer feedback groups, students from other groups can also look at the feedback of these groups, so tutors could ask them to look at good examples of feedback comments in particular files if they chose, in order to glean further ideas about how to write this.

Another benefit of discussing the peer marking in this way is that it helps students to practise collaboration skills. Furthermore, the peer feedback review lessons which follow the written comments can enable students to practise negotiating and discussing in English if they do this in class time. These are skills which they will need for their assessed discussions, which form 15% of their final marks. A further benefit of being involved in this process is that it encourages students to critically analyse academic writing and thus may help to improve their critical reading skills.

The process

The original aim of the peer feedback activities was for three student peers to provide written feedback on three areas, content, organisation and language, while the tutor would only provide oral feedback on these. However, there was more general discussion on how to achieve this. Several tutors mocked up solutions in the shared class notebook used for developing materials and the merits of each were discussed. It was agreed that the peer marking should be conducted in the collaboration space, to be easily accessible to changing groups of students and the external examiner. Within each class, students were divided into peer marking groups of three to five (depending on class size), and within these peer marking groups each student was expected to mark three peers. Different layouts were suggested, and the potential benefits of having one section per student within the collaboration space or one section per task were discussed. One solution involved tutors putting a page for each group of three to four students in the collaborative space, while another solution was to simply include a page for each student and an accompanying matrix of the peer marking groups for that task. This approach had the benefit that the section could be reproduced for subsequent tasks with the groups simply being changed in the matrix. While different tutors took different approaches to the setting up of peer marking groups within class notebook, most tutors made use of the same template created by one tutor.

For each task set, students were asked to first write their work in their individual writing portfolios, then copy this over to the table in the collaborative document SWJTU Leeds tutor peer feedback template. Each student was asked to add comments on the work of other students in the group in answer to the questions. In most classes, the peer feedback groups were changed after each writing task was completed, so that students had the opportunity to give feedback to and receive this from different peers. However, in one lower-level class, the tutor asked students to mark just one other student per group for two tasks, due to students’ workload on other courses and to encourage them to spend more time giving detailed feedback. In addition, in at least one other group, the tutor was approached after the first task with a request that the group remain the same. The students mentioned trust built up within the group and fears that certain other students may not be as diligent in giving feedback as peers in their original groups. When it was explained that the rotation of groups was designed to prepare them for workplace situations where they would need to work collaboratively in different project teams, not with their friends, the change was accepted and appeared to work well in most groups.

The peer feedback itself involved asking students to comment on which areas of content, organisation and language their peers had done completed well and which they needed to improve on. For the first two feedback tasks, students were encouraged to focus on features of content and organisation, as these are elements which they typically do not notice errors in. For instance, they were asked to indicate whether the question was fully answered, and the content was relevant, as well as whether students’ answers were logically organised into paragraphs. It was also suggested that for the first two drafts, students highlight just a few examples of linguistic errors in content and structure for their peers. The students were not given an error correction code or asked to use error correction symbols until the third assignment, so that they could focus on identifying general linguistic errors, even if they did not know what they were. However, from Task 3 onwards, as well as continuing to comment on content and organisation, groups were introduced to the error correction code and asked to find errors in their peers’ work or put correction symbols next to these, with some tutors stipulating a minimum number of errors to find. Some tutors also asked students to focus on different areas of language, such as passives and relative clauses, which they had studied in lessons that fortnight, so that they focused on different linguistic errors in different writing tasks. After each of the five assignments, tutors continued to review each essay and its peer feedback and keep a record of comments that would assist in marking the final version of the portfolio, including areas in which students had improved.
For this to work as intended, it was important to provide clear instructions. Students were therefore given instructions on where to write, who to work with and whose work to mark before beginning their first feedback assignment. They were also shown how to copy their documents from their class notes to the collaboration space during class time, after submitting their first draft of Task 1, so that the tutor could check that all the drafts were shared in feedback groups before students began writing the feedback. Some tutors created tables with lists of which students would work in which groups. This document was shared in the collaborative space in Class Notebook above the peer feedback pages, as well as in the class Teams channel, so that students could easily check who they were working with. Providing such instructions before students started to complete their first peer feedback task therefore enabled them to complete the task independently without having to ask many questions and meant that they knew what to do for the subsequent peer feedback task.

After each draft was completed, tutors conducted a lesson to review the peer feedback. Some tutors saw each group in a separate meeting and facilitated a session on each group’s peer feedback, providing questions for them to discuss at the start of the session before adding their own comments on the peer feedback. Tutors conducted the sessions of approximately 20 minutes for each group of three to five students, depending on class sizes, in slightly different ways. Some tutors elicited what students had found useful or not so useful about each other’s feedback and made some suggestions for how to improve the content or detail in this feedback.

How it worked in reality

There were some notable differences between how tutors expected these sessions to work and how successful these initially were. In terms of the written peer feedback, some tutors found that most students completed the peer-feedback tasks, while others found that not all students did so. Some students therefore needed reminding to do this. The level of detail and usefulness of the comments also varied. At first, some students identified everything as good, but some tutors elicited why such feedback might not help students improve and most students were then able to give more critical feedback where needed for subsequent tasks. Another area which students sometimes misunderstood was how to use the error correction symbols or pick out certain types of error. One tutor provided some asynchronous tasks on identifying the meaning of error correction symbols provided by the course leaders. Students then spent part of a lesson practising how to identify certain types of error and using the error correction symbols in sentences in order to scaffold the approach to using these. However, instructions on this needed reinforcing as many students tried to correct work for others, rather than just using the symbols. It was therefore sometimes necessary to remind them not to do this, eliciting the fact that the work should be corrected by the student who had written it. Despite these issues, some students provided some insightful comments which their peers mentioned helped them to improve.

In general, students’ ability to give constructive feedback on content and language improved with each peer feedback assignment. Students were given the chance to be involved in the same process multiple times and scaffolding this by adjusting the feedback template slightly depending on the focus of each writing task. This meant that there was a new focus on a certain area of content or language each time and helped students to improve both their own writing skills and their ability to provide feedback to help their peers improve. It has also enabled them to develop their ability to demonstrate critical thinking, since they need to evaluate the usefulness of the feedback they had given or that had been given to them by other students. Tutors also found this a useful method of encouraging students to work together asynchronously, while also being able to monitor this. However, when looking at feedback on the most recent draft, one tutor noted that despite having had lessons with both themself and the SWJTU teachers in China, students did not always pick out issues with paragraphing. This was useful in serving to highlight an area which they needed more work on, which they did in the subsequent feedback session.

The feedback sessions conducted by the tutor on how students had completed the feedback did not always run as intended. Having done small group essay feedback sessions in face-to-face scenarios, some tutors mentioned that they initially expected this to work in the same way. In other words, they envisaged that small groups of students would all listen and ask questions on the points raised in each student’s work. To facilitate this and to avoid the feedback noises of multiple devices in the same space, some tutors arranged for each group to log on with a single laptop and sit around this laptop for a group discussion. In reality, this resulted in each student taking the hotseat as the tutor discussed their essay with the rest of the group disappearing into the background, passively listening and very few questions were asked, so this became a sub-optimal learning activity. In the subsequent staff meeting it became clear that many of the tutors had found that “giving oral feedback” was not the best way to build on the peer feedback. The subsequent discussion led to a variety of strategies in later sessions.

From the second task onwards, techniques for reviewing the peer feedback included the following. Some tutors picked one or two points from each student's essay about what students had done well and areas to improve on for each task, both in terms of giving feedback and in their drafts. These varied between indicating significant language or structural points if none of the peers had noticed these, points that had erroneously been commented on by peers and specific complex sentences that had multiple issues for unpicking.

The small group sessions were now structured as a discussion between the students, with the tutor as observer. Before the session, students were asked to review the feedback which they had received, and to consider the quality and usefulness of that feedback. This formed the starting point for the group discussion. In addition, this had the benefit that any students that were not contributing to the process were immediately visible. Certain tutors then introduced the specific points that they had selected, but rather than “giving feedback”, they asked the group questions. This meant that the students were engaged in active learning in a collaborative manner, while the connection with specific points from the students’ essays meant that each student also felt they were getting “tutor feedback” on their work.

Lessons learnt and areas for improvement

Even though most students generally improved their peer feedback and writing skills, there were some areas in which the tasks could be improved. In future courses, it would therefore be beneficial to concept check instructions on what to do for each feedback task more clearly before students complete the task. This does not always seem to work so well if individuals are simply asked for answers or to write in the chat box on Teams (our class platform for this course). Creating a quiz for students to do, either asynchronously or in class, in groups on Microsoft Forms or through Kahoot, would enable them to check their understanding of the instructions. Some questions from earlier quizzes could also be reframed if students have not completed some of the previous tasks in the manner suggested. In addition, it would be beneficial to give students an example of a peer marked task on a different subject from the ones for assessment, so that students can answer questions on the usefulness of specific types of comment and discuss these in a lesson before the first feedback task is set. This would give students some concrete ideas about the level of detail and type of comments required.

When completing the task, some students were better at doing this than others. Most did as they were asked, but a few people needed to be reminded to complete the tasks set on one Friday by the Friday of the following week. This was so that tutors also had time to review the writing and peer marking before the peer marking review lessons the following Tuesday. It was also necessary to remind some students to complete the re-writes, though by the time of the fourth submission they generally did this without asking. In addition, in the peer review sessions, some students were not very responsive to questions about what they had or had not understood from their peers’ feedback, or how useful this was. Therefore, as the term progressed, the strategy of supplying a few reflection questions for students to consider how useful their peers’ feedback had been before the class, in preparation for the live feedback session, became more common among the tutors.

In terms of technical issues, certain problems that occurred in earlier experiences of lockdown remote teaching were avoided, as OneNote for this course has been set up separately from Teams. However, unlike SharePoint or OneDrive, there are occasional synching problems if several students try to look at these at once. It is therefore worth ensuring that students are aware of how to delete previous versions of the page where necessary. Some students also had issues with formatting their documents according to the formatting guidelines for the task, resulting in large spaces between lines and long thin texts which made these hard to read. It could therefore be advisable to tell students to write their work in class notebook to begin with, rather than saving this on their devices beforehand, so that they can use the correct format here. Students also need to be aware that they can widen columns in Class Notebook, so that their text is not too long and thin and is easier to read.

Overall, conducting peer feedback in this manner has proved useful both in terms of improving students’ writing and collaborative skills, as well as encouraging them to develop some of the skills which they need to become independent learners. While this teaching team comprised experienced teachers with individual teaching styles, experimenting with different strategies in an element which was new to the course this year, these were discussed in the weekly team meetings, frequently leading to collaborative solutions to shared problems. In future, it might therefore be beneficial for the teaching team to agree on a common scaffolded approach focusing peer feedback on different areas for each writing task, taking the most useful elements from the approaches adopted by different tutors.

We look forward to receiving your comments. If you have any if you have any suggestions about the SWJTU-Leeds Joint School, please contact Joanne Shiel at: J.R.Shiel@leeds.ac.uk.